Decision-Making in Artificial Intelligence: Is It Always Correct?

Big data and artificial intelligence (AI), which apply big data, are still actively discussed in the medical field. Diverse studies utilize big data with the goal of developing AI,1, 2, 3 and there have been various attempts to apply such studies in the clinical area.4, 5 In such an environment, the following questions are often posed by physicians. “I'm a medical doctor, (or nurse) my field is completely unrelated to information and communications technologies (ICT), but do I still have to study these things? Am I falling behind if I don't?” We do not know what the correct answer to that question would be. Of course, AI could play a meaningful role in academic research. After all, medical practice is the medical science of making medical decisions, followed by some type of medical action.

Most physicians follow best practice protocol when making therapeutic choices, while drawing on their rich personal experience and the patient's informal hints to reveal additional information about their condition. However, this aspect of diagnosis is often not systematically recorded. In the long term, training is aimed at developing inexperienced physicians into confident clinicians who can interpolate missing information to detect the existence of underlying symptoms or conditions. There is a standardized protocol for diagnosis, as well as action guidelines based on diverse medical situations.6, 7, 8 Of course, when observed more closely, there may be diagnoses that violate the guidelines and treatments that are incorrect according to the protocol.9 However, do physicians apply incorrect treatments because they do not know the proper guidelines and protocols? When physicians make medical decisions in an actual hospital setting, ethical or medical principles that are not included in data-based guidelines and protocols come into play. As soon as we forget to consider this part of the decision-making process, we fall into the delusion of AI's grandeur.

Medical practice consists of four steps: screening, diagnosis, treatment, and monitoring (Personally, I think the screening step could be supported by AI as it seems to provide a solution for the shortage of staff in the medical field). However, medicine does not rely only on the precision of these steps. Modern physicians provide diagnoses and treatment based on scientific examinations that are evidence-based, objective, and standardized; they accept this as the natural order of things. However, their decision-making is quite heavily influenced by external data rather than data from within the hospital. Even if it does not follow the guidelines of a textbook, diverse diagnoses and less-effective treatments can be attempted when deemed necessary, as long as they are subject to strict monitoring.9 Treatment may be postponed due to the patient's financial situation. Depending on the patient's will to live, a physician may attempt impossible or less successful treatments. From the perspective of big data and AI, such actions may seem irrational or incorrect.

When we discuss the issue of decision-making in AI, the fundamental questions to be answered are not whether AI provides better judgment than physicians, or even whether the judgment itself is right or wrong. When one patient with a strong will to live has a survival probability of 0.01% and another suicidal patient has a survival probability of 99.9%, who should be prioritized? Let's hypothesize that AI data predictions reveal a patient has a 99.9% chance of dying or 99% chance of getting a fatal type of cancer after six months (I do not know if the patient knowing their future will be beneficial to their life). If AI concludes that treatment is meaningless or unhelpful due to such probabilities, is this conclusion valid? How will we deal with the issue of euthanasia? AI may suggest various treatment steps and methods deemed appropriate for such patients, but this is precisely when big data overreach its good intentions. For AI, making the most rational decision with the highest level of accuracy is the priority; it is not interested in good intentions. But in reality, the best decision may not be according to the order of priority. Finally, big data are expected to bring immense innovation to the medical field, but decision-making is ultimately the most important factor in this field. We must approach AI with a clear understanding of such matters. In other words, our judgement is that we must be free from the dichotomous thinking of AI's choosing between ‘0’ and ‘1’.

Ultimately, the need to incorporate social values in big data is an essential AI requirement.10, 11, 12 Therefore, AI should consider legal aspects, human dignity, and hope; social values must be taken into account as opposed to merely finding the outcome with the highest success rate from the data. Eventually, we also want AI to incorporate our conceptualizations of ethical treatment—something that is currently not easily achieved. What must we consider to be important when studying big data or AI in the medical field? As mentioned above, the essence of AI is ‘medical decision-making’. At this point in time, it may play a ‘decision-supporting’ role for physicians. In any case, it is essential that AI plays a key role in the decision-making process.

When considering these aspects, medical data from within the hospital can be supplemented by the implementation of AI in the medical field more broadly. In conclusion, I think big data and AI can be utilized as supporting tools for the decision-making process of physicians. More data do not necessarily equate more valuable data, and we must recognize the importance of value in the medical decision-making process to adopt an algorithm with flexible values. Physicians will have to continuously think about the standards of data reliance and medical intervention. Ultimately, the physician is the final decision-maker. There is no need for physicians to be swayed by AI, but improved, evidence-based reliability of AI-informed diagnoses must be constantly pursued in an academic context. However, considering the impact of technologies such as AI on the medical community, it is also necessary to embrace technological change, and we need to have the capacity to continuously learn, adapt and apply new technologies. AI can provide the best predictive results in the environment it was trained. The important point is that physicians need to have sufficient understanding before using it, as well as awareness of factors that were not considered in training process, when making a decision. The final decision should be made by medical experts able to flexibly consider all these factors.