campusreview. com. au
INDUSTRY & RESEARCH not clear. Many of the relevant policy documents are not publicly available. This itself is clearly a problem. Quoting the Roman poet Juvenal, Quis custodiet ipsos custodes? Or‘ Who guards those on guard?’ Despite the lack of transparency, much has been uncovered about ERA procedures, which I’ ll describe below in some detail. Universities know where the loopholes are, and can take advantage of them. Games one can play to cheat the ERA fall into four categories:
FIELD OF RESEARCH GAMES
1 The ERA divides research into Field of Research categories, determined by the standards for New Zealand and Australia. Yet the universities decide which category research papers go into.
Problem: Universities can‘ improve’ the FOR system, to their benefit, by placing highly cited papers into FOR areas where they want a strong result, and hiding others in fields where they don’ t do much research. This is a university doing what should be the job of the ERA.
Solution: The ERA must remove this conflict of interest. The truth is that if a paper’ s Field of Research is ambiguous enough that the Australian Research Council can’ t determine what it is, it probably is not important in an increasingly diverse world. In any case, classification should be determined externally. Don’ t set the fox to guard the henhouse.
NEW STAFF GAMES
2 One ERA peculiarity is that when a staff member changes workplaces, it attributes that person’ s previous research to his or her new institution, even though all that work was done elsewhere.
Problem: The new staff member’ s work may not involve Australian research at all. This allows cynical universities to buy a track record without publishing anything. A good university or research institute must provide an entire ecosystem of research excellence, which must be evaluated in situ.
Solution: If a university has its address on a paper, then it deserves the relevant research credit. When the research is done somewhere else, it should not be included in ERA assessments.
Don’ t judge a book by its cover, nor a university by what its staff did elsewhere.
3MULTIPLE AUTHOR GAMES Research papers today often have long author lists. The ERA credits a full publication to each of the co-authors – but only when they are not at the same university.
Problem: Modern collaborations have up to 5000 or more co-authors. The ERA treats the output of major collaborations such as CERN as though they are carried out entirely at just one university. This over-estimates the actual research produced at Australian universities by a factor of up to 5000.
The justification for this procedure is that attributing the output of a large consortium to just one co-author encourages collaboration. This is not the goal of the ERA. It is set up to measure research output objectively. Making decisions requires accurate and truthful data. If the data is distorted for other purposes, good or bad, it ceases to have the same utility, defeating the purpose of the ERA.
Solution: Very simple. If a scientist writes half a paper, they are credited with half the work. This is surely all they are entitled to claim, and all the ERA should be counting.
Let’ s count only research that we do, not research that others do.
FAKE JOURNAL GAMES
4 The last decade has brought an explosion in the numbers of fraudulent journals, which are now in excess of 50,000, and increasing rapidly. The ERA dropped its original scheme of journal quality ratings, which would have dealt robustly with this, and now has no mechanism to exclude fakes.
Problem: Publishing in a fake journal requires only payment of the publication fee. These publications claim refereeing takes place, but it’ s perfunctory at best. This has little to do with quality in science.
More generally, there is a continuum of quality in science publications, from short micro-papers that may never be cited, all the way to authoritative and highly cited reviews. Yet counting citations does not measure quality. Fake journals cite other fake journals to build an alternative universe of completely fraudulent science, which has no place in the ERA.
There must be a reliable way to ensure that low-quality, pay-to-publish outputs are excluded.
Solution: Either bring back journal quality rankings, which were better than nothing, or at least use the authoritative Beall’ s listing of fake publishers to eliminate low-quality papers.
If we want excellence, we have to eliminate fraud.
SUMMERKORN’ S CHOICE Let’ s put it all together, from the perspective of an imaginary university administrator, Dr Summerkorn, charged with upgrading the research output at Blackwater Stump U. There are two ways to do this: work hard and spend money to do research in some target FOR, or install a defeat device, which to Summerkorn seems much easier.
A. The hard route Blackwater advertises for new research staff, buys research equipment, pays start-up costs, and tries to recruit good graduate students in a chosen, strategic FOR. ERA ratings eventually pick up. This is a slow process, naturally. It’ s also expensive, and results are seen only long after Summerkorn has gone on to fresh pastures.
B. The easy route Summerkorn recruits several retired consortium participants. Existing, low-impact papers at Blackwater Stump U. are lost using loophole( 1). Loophole( 2) adds prior research of new staff to the university’ s reputation. Loophole( 3) amplifies the credit for consortium papers a thousand-fold. And finally, loophole( 4) allows fake papers – all citing one another – to go into the mix.
It’ s noteworthy that this easy route at no stage involves carrying out any actual research in Australia.
NO MORE GAMES The easy route is the proverbial lipstick on a pig. It’ s not suggested here that any university uses all these loopholes. Summerkorn’ s job at Blackwater Stump is purely fiction, for now. And given the secrecy of the ERA, it’ s even possible that the loopholes are already blocked. If so, the ARC is to be applauded, but there is no evidence to suggest this has happened.
In summary, there is an urgent need to improve the ERA – to make its process as excellent as our research should be. ■
Professor Peter Drummond is the science director, Centre for Quantum and Optical Science, Faculty of Science, Engineering and Technology, Swinburne University of Technology.
17