Assessing and quantifying education standards is intrinsically difficult, as anyone working in the field should tell you. As such, proclamations made when the results of an assessment are published must be taken with a pinch of salt - read The Tiger That Isn't or Bad Science for examples of meaningless statistics used to make outrageous claims. It's not that we shouldn't try to examine and compare educational institutions, but that the nature of the assessment should be made clear.
British universities recently underwent the latest Research Assessment Exercise (RAE), theoretically enabling people to decide which departments are the best for research in different fields. I was a PhD student at the University of Birmingham when the 2001 RAE took place, and the geology department was deemed to be of insufficiently high quality. Not enough internationally excellent research was being carried out, so the government funding was cut, numerous staff members were made redundant, and the geology department was subsumed into a new School of Geography, Earth and Environmental Sciences.
7 years on, I'd become a postdoc at the University of Aberdeen, conducting my own research and therefore seemingly in need of scrutiny under the 2008 RAE. However, although I had published a sufficient number of peer-reviewed articles to be included, the powers-that-be decided my work was not high-profile enough to warrant submission. I cannot argue that this decision was wrong - my papers about weird fossils are of interest to few people - but it does undermine the value of the subsequent RAE results. If only some research is submitted for review, then the assessment can only apply to the department (or university) in partim. Instead, the results are out, and the University of Aberdeen is able to claim that "89% of Aberdeen’s research activity is of international quality".
This is misleading. What it should say is that "89% of the research that the University of Aberdeen chose to be submitted for assessment is of international quality", but this is omitted from the press releases. It is left to the uninitiated reader to assume that the entire university has been assessed for research quality.
To be fair to my former employers, their approach is certainly not unique. Every university across the country will have done exactly the same thing, so ultimately the RAE results will be equally meaningful across the board. It just seems a pity that the fear of 'failure' requires such games to be played, although I'm told the RAE system is to be changed, so perhaps this won't happen next time round.
British universities recently underwent the latest Research Assessment Exercise (RAE), theoretically enabling people to decide which departments are the best for research in different fields. I was a PhD student at the University of Birmingham when the 2001 RAE took place, and the geology department was deemed to be of insufficiently high quality. Not enough internationally excellent research was being carried out, so the government funding was cut, numerous staff members were made redundant, and the geology department was subsumed into a new School of Geography, Earth and Environmental Sciences.
7 years on, I'd become a postdoc at the University of Aberdeen, conducting my own research and therefore seemingly in need of scrutiny under the 2008 RAE. However, although I had published a sufficient number of peer-reviewed articles to be included, the powers-that-be decided my work was not high-profile enough to warrant submission. I cannot argue that this decision was wrong - my papers about weird fossils are of interest to few people - but it does undermine the value of the subsequent RAE results. If only some research is submitted for review, then the assessment can only apply to the department (or university) in partim. Instead, the results are out, and the University of Aberdeen is able to claim that "89% of Aberdeen’s research activity is of international quality".
This is misleading. What it should say is that "89% of the research that the University of Aberdeen chose to be submitted for assessment is of international quality", but this is omitted from the press releases. It is left to the uninitiated reader to assume that the entire university has been assessed for research quality.
To be fair to my former employers, their approach is certainly not unique. Every university across the country will have done exactly the same thing, so ultimately the RAE results will be equally meaningful across the board. It just seems a pity that the fear of 'failure' requires such games to be played, although I'm told the RAE system is to be changed, so perhaps this won't happen next time round.
Comments