Or what ChatGPT and artificial intelligence could mean for the future of medicine

Without devouring a unique didactic book of the area or after passing a day at the Faculty of Medicine, or co-author of a study In the pre-print phase, I answered a series of practical questions. The number of correct answers was enough to pass the exam that enables doctors in the United States.

But the examinee was not a member of Mensa (organization that studies the gifted) or a savant of the health area, and just artificial intelligence ChatGPT.

Created to answer the questions of two users in the form of a conversation, the tool has generated such a stir that doctors and scientists are trying to determine what are its limitations – what it can do for health and medicine.

What is it or ChatGPT – or what is it not

ChatGPT, or Generative Chat Pre-Treated Transformer, is a natural language processing tool powered by artificial intelligence (AI).

The technology, created by OpenAI, based in São Francisco, and launched in November, is not like a search mechanism with good speech. Ela sequer is connected to the internet. Instead, a human programmer feeds a large amount of dice online that are maintained on a server.

AI can answer questions even though it has never seen a particular sequence of words before, because the ChatGPT algorithm is trained to predict which word will appear in a sentence based on the context of what it sees before. Ou seja, it is based on the fact stored on your server to generate your response.

The ChatGPT can also answer questions in sequence, admit errors and reject inappropriate questions, depending on the company. It is free while it is in the testing phase.

The interest in the medical area is large

Artificial intelligence programs have not existed for some time, but they are of such interest that medical organizations, professional associations and magazines in the field are trying to figure out how they can be useful and understand what limitations and ethical concerns they can bring.

The clinic of Dr. Victor Tseng, Ansible Health, created a force-task to deal with the matter. A pneumologist, Dr. Tseng is the medical director of the group based in California and co-author of the study in which ChatGPT demonstrated that he could probably pass the medical admission exam.

He told him that his colleagues started jumping around with ChatGPT not last year and were intrigued when the program accurately diagnosed fictional patients in hypothetical scenarios.

“We were so impressed and truly discouraged by the eloquence and kind of fluency in his response that we decided to use it in our formal review and test process by reference to medical knowledge,” Lembrou.

The test was applied to the three parties that graduates in medicine in the US have to pass to be licensed for medical practice. The exam is considered one of the most difficult of any profession because it does not ask simple questions with answers that can be easily found on the internet. In addition to testing basic science and medical knowledge and case management, it tests clinical reasoning, ethics, critical thinking, and problem-solving skills.

The study team used 305 publicly available test questions in June 2022, when none of the answers or related context had been indexed in Google – and, therefore, did not provide information about as quais or ChatGPT was trained. The authors of the study will remove examples of questions that have images and graphics and start a new chat session for each question they ask.

Some students usually spend hundreds of hours preparing and the faculties normally give them a long time from the classrooms just to study for the test. The ChatGPT does not require any preparation work.

IA passed or almost got all the parts of the exam right without any specialized training, showing “a high level of concordance and perception in their explanations”, wrote or studied.

Impressed doctor

“The general exam contains many false clues,” he said. “It is very difficult to do a good job or try to discover intuitively with an approach like that. A person can take hours to answer a question of this manner. But ChatGPT managed to give an accurate answer about 60% of the time with convincing explanations in five seconds”.

Dr. Alex Mechaber, vice-president of the US Medical Licensing Exam at the National Board of Medical Examiners (Conselho Nacional de Examineros Médicos), says that the results of ChatGPT approval are not surprising.

“The content is quite representative of the medical knowledge and the test item the type of multiple questions chosen as quais to IA will probably soon happen”, opinou.

The Mechaber doctor says that the council is also testing the ChatGPT does not test. The members of the council are especially interested in the answers that IA missed to understand or why.

“It’s an embarrassing technology. We are also very aware of the risks that the great models of language trace in terms of potential misinformation, stereotypes and preconceptions ”, he affirmed. “I hope that this technology will get better and better and we are excited to discover how we are going to embrace it and use it in the right way”, he said.

Other medical possibilities

The ChatGPT has already entered the discussion about research and publication.

The results of the study of the medical license exam were written with the help of ChatGPT. The technology was originally listed as a co-author of the research, but Dr. Tseng says that, when the study is published in the journal “PLOS Digital Health” this year, ChatGPT will not be listed among the authors because it would be distracting.

No more past, to magazine “Nature“Creou guidelines that say that no program of this type could be credited as an author, because “any attribution of authorship entails responsibility for the work, or that the AI ​​tools cannot assume.”

Another article recently published in the “Radiology” magazine was written entirely by ChatGPT. The text questioned by a human author could be substituted by the machine, and the program is ready for many of its possible uses, including writing study reports, creating documents for patients, and translating medical information in a variety of languages.

Still, there are limitations.

“I hope that technology will help, especially if AI requires security support, a correction,” said Linda Moy, editor of the magazine “Radiology” and professor of radiology at the Grossman School of Medicine at NYU.

According to her, the article from ChatGPT was quite accurate, but she created some references.

Another medical concern is that the AI ​​could make dice. AI is so good how much information it feeds with – and, with so much inaccurate information available online on topics like vaccines against Covid-19, it could generate inaccurate results.

Artie Shen, a colleague of Dr. Moy and a fellow at the New York University Data Science Center, is exploring the potential of ChatGPT as a kind of translator for other AI programs for medical image analysis. For years, scientists will study AI programs from startups and larger operations, like Google, that can recognize complex patterns in image data. The hope is that they will provide quantitative assessments to potentially discover diseases more effectively than human eyes.

“AI can give a very precise diagnosis, but it will never say how I got that diagnosis”, Shen opined. He believes that ChatGPT can work with other programs to capture your logic and observations.

“If we can talk, there is a chance that these systems transmit their knowledge in the same way as an experienced radiologist”, he said.

A tool to improve doctors?

Dr. Tseng said that, in the last analysis, ChatGPT can improve medical practice in the same way that online medical information has strengthened patients and forced doctors to communicate better, because now they have to provide information about what patients readem online.

The ChatGPT will not replace the doctors. Tseng’s group will continue to test it to learn why it creates certain errors and which other ethical parameters need to be set in practice before using it in real life. More or less a doctor than a tool can make the medical profession more accessible. For example, a doctor might ask ChatGPT to simplify a complicated medical jargon in a language that someone who hasn’t finished or taught basics understands.

“AI is here. The doors are open. My great hope is that it will really make use of our best doctors and service providers”, affirmed Tseng.