It is worth stressing here that ChatGPT has not been specifically

engineered to trawl datasets

restricted to professional medicine ; unlike Med-PaLM , it ’ s rummaging through the internet for its responses .

But Dr Tamayo-Sarver said the issue goes back to the fundamentals of the technology , which goes back to the absence of any semantic understanding of what it generates .

“[ LLM ] AI is really noting the association between words and has no underlying conceptual model …

“[ But ] in the medical diagnosis use case , the first step is creating a well-articulated problem statement of what is going on with the patient , which requires a lot of problem solving , which requires a conceptual model , which [ an ] LLM does not have .”

Medical questions and actual medicine

Doctors will also no doubt welcome

the observations of Professor James Davenport in response to the Nature article on the abilities

of Google ’ s Med-PaLM .

He said the press release issued by Nature was “ accurate as far as it goes ”, acknowledging that the paper advances knowledge of using LLMs to answer medical questions .

“ But there is an elephant in the room , which is the difference between ‘ medical questions ’ and

actual medicine ,” the professor of information technology at the University of Bath in the UK said .

“ Practising medicine does not consist of answering medical questions — if it were purely about medical questions , we wouldn ’ t need teaching hospitals and doctors wouldn ’ t need years of training

after their academic courses .”

Professor Anthony Cohn , from The Alan Turing Institute in the UK , was also wary .

“ The problem of ‘ hallucinations ’ that LLMs are prone to has been widely reported in the literature … Because of the statistical nature of LLMs , it is likely that this

will always be a problem , and thus LLMs should always be regarded as assistants rather than the final decision-makers , especially in critical fields such as medicine .

“ Indeed , ethical considerations make this especially true in medicine , where the question of legal liability is also ever present .

“ A further issue is that best medical practice is constantly changing and the question of how LLMs can be adapted to take such new knowledge into account remains a challenging problem , especially when they require such huge amounts of time and money to train .” And Dr Google ? Dr DeSalvo told The Guardian : “ One of the things that we ’ re really focused on at Google is the tuning of the model and the constraining of the model in such a way that it leans [ to the ] factual .

“ Whether it ’ s for a clinician or for the patient , you don ’ t want a sonnet about your chemotherapy , you want to know [ what ] the literature [ says and ] is that right .” Nature 2023 ; 12 Jul .