Abstract
Humans have devised machines to replace computation by individuals since ancient times: The abacus predated the written Hindu–Arabic numeral system by centuries. We owe a quantum leap in the development of machines to help problem solve to the British mathematician Charles Babbage who built what he called the Difference Engine in the mid-19th century. But the Turing formula created in 1936 is the foundation for the modern computer; it produced printed symbols on paper tape that listed a series of logical instructions. Three decades later, Olivetti manufactured the first mass-marketed desktop computer (1964), and by 1981, IBM had developed the first personal computer. Computing machines have become more and more powerful, culminating recently in Google’s claim that it had achieved quantum supremacy in developing a system that can complete a task in 200 seconds that it would take the most powerful type of classical computer available 10 000 years to achieve. In short, we are in a period of human history in which we are creating more and more powerful and complex machines potentially capable of duplicating human intelligence and indeed surpassing/expanding its power. We are solidly in the age of artificial intelligence (AI). Increasing interest in the development of AI and its application to human health at all levels makes a roundtable discussion by experts a valuable project for publication in our journal, Gender and the Genome, the official journal of the Foundation for Gender-Specific Medicine and the International Society of Gender Medicine.
Keywords
The state of AI hype has far exceeded the state of AI science, especially when it pertains to validation and readiness for implementation in patient care. Eric Topol
1
Good morning.
Let’s begin our discussion with Doctor Simon’s views on the burden of collecting and processing/interpreting large masses of data with the aim of individualizing and focusing patient care. In particular, we are concerned with ensuring the accuracy of diagnoses made by computer is at least as accurate as that made by trained human observers. Some limitations of the current uses of such systems include the use of nontransparent algorithms for patient care, the inequity of the application of successful AI to disadvantaged societies/persons, and insuring confidentiality of the data.
First, let me cite Eric Topol’s definition of artificial intelligence (AI). 1 It is the science and engineering of creating intelligent machines that have the ability to achieve goals equal to that of humans via a constellation of technologies.
I would like to discuss the transformation of health care on 3 levels: clinical trials, point of care, and coaching patients about their health. From the laboratory to the market, the principal stages are, as you all know, drug discovery, clinical trials, clinical practice, point of care, and patient coaching after the encounter with the physician. In research, drug discovery is being revamped by AI in several ways: data mining, natural language processing to extract relevant information from unstructured text such as physician notes, and sorting multiple databases to support personalized medicine. This has yielded mixed results.
In clinical trials, AI may help identify relevant patient populations and support remote monitoring via biosensors. In clinical practice, AI is beginning to be widely used, including in the rapid analysis of radiologic images. Finally, at the point of care, there are several areas under development. One is the concept of a virtual scribe where AI would act as an assistant to the doctor during a consultation. This is less invasive than a human scribe. But here, physician acceptance varies by specialty; in general, there is a very limited acceptance of the whole concept of the use of AI in clinical practice. For instance, a survey by the consulting firm ZS showed that 77% of doctors accept AI as a tool, 43% as a diagnostic advisor, but only 18% accept AI as a peer diagnostician. This was true across all specialties.
Dr Young, do you have any feelings about the acceptance of AI by the medical or research community for development of tools for assistance in diagnosis and suggestions for treatment?
My initial thought was that I am quite encouraged by this resistance. I feel that doctors and health-care professionals are appropriately skeptical. I work with robots, and we have seen that we don’t know yet exactly how these things—robots and intelligent AI systems—will impact people. We have seen evidence that people may over-trust such systems and that technologies can be easily designed to be persuasive or manipulative. Given what Dr Simon said, I wonder if doctors may help people regard AI devices in a healthy skeptical fashion. I do feel the resistance may give the community time to catch up and learn more about how we can and should present the data from robots and AI systems to people in a way that can support integrating AI into the whole care package, thus avoiding problems such as over-trust, reliance, or persuasive and manipulative behaviors.
Dr Sánchez-Serrano, can you talk about the use of AI in the pharmaceutical world?
Thank-you very much; I am very honored to be part of this panel. The work that I have been doing over the past 16 years has been to understand the relationship between innovation in the drug industry and the global delivery of high-quality care to the patient, regardless of whether they are in industrialized countries or in the developing world. The world’s health-care crisis is truly a global and multidimensional problem. So, I have been interested in finding ways for innovation and technology to be most effectively translated into health-care commercial products. For these reasons, I wrote a first book about the specific role of the pharmaceutical industry within the global health-care system. 2 Recently, I have published a second book on an organizational paradigm, which is an economic theory called the Core Model. 3 This model offers insights into how scientific collaboration happens in a specific structure, which if done correctly could save time, labor, and economic resources when developing a new drug. Artificial intelligence can benefit from this model because the collaborative nature of this technology is in total accord with the Core Model and could save time, resources, human effort, and money when applied to the process of drug discovery and development.
Artificial intelligence can contribute greatly to the improvement of patients’ access to the right medication, detect gaps in care, identify unique needs, and avoid duplication of effort through associating the patient with the patient’s environment, geography, sex, age, nutrition, occupation, and overall life style.
I am working now on drug discovery in the field of genetic disorders and neurological diseases which are gender-specific. Artificial intelligence can also be applied to finding the proper market in which the medication will be used. This could solve the problem of direct-to-consumer advertisement that greatly increases the cost of medication.
Drug discovery and development consists of 4 stages: target identification and validation, lead compound identification and optimization, preclinical studies, and finally phases 1, 2, and 3 of clinical trials. In each of these, AI is being currently used. For example, in primary drug screening, AI image recognition and robotics are being sued for cell target identification, diagnosis, and sorting. The AI-based convoluted deep neural network algorithms have proven to be highly sensitive and specific for high-content sorting of Chlamydomonas reinhardtii and human platelets.
In secondary drug screening, AI is being used in the prediction of important physical characteristics (such as melting point, atomic composition and subatomic behavior molecular weight, solubility, etc) that are crucial for drug design or drug selection, as these characteristics determine bioavailability, bioactivity, toxicity, and so on. There are a significant number of algorithms (ie, Deep Chem, Deep Tox, DeepNeuralNet-QSAR, etc) that help determine and predict bioavailability, bioactivity (quantitative structure–activity Relationship Analysis), and toxicity. Artificial intelligence–based pharmacodynamics and pharmacokinetics modeling is currently utilized. It is also used in the prediction of the 3D structure of a target protein, drug–protein interactions, protein engineering, and gene expression data analysis, as well as in the automation of chemical synthesis. There is a small handful of commercial drugs that have been discovered in the last 7 years using complex computational methods.
Artificial intelligence can also trace the complex biochemical pathways that a given drug can take when metabolized in the liver. Two-step metabolites, which are significantly more difficult to detect experimentally than direct metabolites, can become very toxic, even lethal to humans but can be detected with AI. This was the case of the antifungal drug terbinafine (Lamisil; Novartis, Basel, Switzerland) which produced a 2-step metabolite called TBF-A, which is responsible for the liver toxicity and even liver failure observed in some patients.
Doctor Sánchez, do you think this provides an opportunity for eliminating pretesting in animals?
I think AI can be of great help in both preclinical and clinical trials, but I do not see animal models or preclinical testing in animals going away. Pharma companies think AI has great potential in this area. My feeling is that it has to be a combination of traditional experimentation and other innovations yet to be developed to run assays. We should never lose sight of the fact that AI, including machine learning and deep learning, is artificial and not creative in the same complex way that the human brain is; AI is about what we program it to perform.
I completely agree with you that AI has an important role in the entire drug development chain. But at this stage, people see AI as a complementary adjunct to what is being done.
I think that is because a large number of people don’t really understand what AI is about, while others are more cautious about its potential. Some even think—and I have had this discussion with several colleagues in the life sciences—that we will eventually create an evil monster that is going to take over the world and subjugate humankind. I think that as time goes by, we will see great developments in AI as well as some limitations. It is just too early to know.
But I would like to explore another dimension: intellectual property rights (IPRs). Since AI is a collaborative endeavor that continues to be built on previous learning, when working in partnerships, what standard guidelines or strategies will be established for those situations in which new IP is generated based on the IP of the partner? I guess that will have to be clearly spelled out in the contracts. One of my conclusions in my book on the Core Model is that IPRs are not really necessary for the effective discovery and development of novel drugs, since nowadays we have different collaborative models for that end, including open-source innovation, among others.
Dr Young, can you comment on what you are doing in designing and using robots for interactions with humans? Are there problems with intellectual rights? Are you patenting these robots? What are your feelings?
I myself don’t engage with the patent process; I don’t have commercial aims beyond research. I would like to focus on one idea that was raised: These devices are artificial and not human. I think that is really an important point. While it seems obvious that machines are not human, there is overwhelming evidence from the human–robot interaction community showing that people tend to give life-like characteristics to mechanical devices that demonstrate any life-like qualities. I’ve noticed some indication that those more educated may be more susceptible because they think they know better than to be fooled, which is particularly relevant in the health-care scenario. Note the famous example of the chess champion Kasparov losing to IBM’S Deep Blue AI, when he said he felt an “alien intelligence.” And so what we see is that while people know these machines are not human, we naturally tend to treat them as if they are. Further, given that people are social, we also tend to apply social interaction norms and standards, including ideas of trust, gender, and authority to robots and AI agents. The problem is well-documented in that giving any hint of human life to an agent—whether it be high intelligence, physical shape, and so on—can change how people interact with them and brings to the forefront a range of social and political issues that are often not considered with machines.
It is not just the data that we have to be careful about but the machines themselves are also biased. The technologies are often the product of white men, and with these devices servicing the general public, we want to make sure we have an appropriately diverse gender-based program. We did a small study in my lab of robot–patient interactions and found that not even half of the papers even reported the gender ratios of the participants in their studies. Of the ones that did report gender, only 1 of 5 studies did an analysis of the impact of gender. So, we have a bunch of men making these tools, and I think that in addition to the data being problematic, the algorithms and machines themselves need to be for a diverse population.
That’s so interesting, because getting biomedical research at any level to include both genders has been an uphill struggle. And it is discouraging how the impact of biological sex, not to mention the impact of the environment on genomic expression, is neglected in the design of research projects.
Dr Nomura, what do you have to say about the interesting aspects of designing robots for human use?
There is a very big difference between AI and patients. Patients have various kinds of feelings about robots. Robots are not human, but they are complex and seem human-like and we want to think about how that changes how people interact with them. In my recent review of the use of robots for the treatment of dementia in the elderly individuals and with autistic children, I have pointed out that the consideration of gender, both in the design of the robot and in the person interacting with it, is an important feature. 4 For example, autistic children are more likely to have eye contact with a robot than with humans. Furthermore, specifically programmed robots seemed more likely to be able to control the obnoxious or disruptive behavior of these children. Male robots are preferred in security scenarios, while a female robot was selected for health-care scenarios. Interestingly, a dilemma arises on the gendering of robots: Such a process may reinforce negative societal views of what tasks are appropriate for a male or female. The design of robots should consider the social and cultural environments in which they will be used; for example, there are cultural views of the tasks robots will be expected to perform.
Dr Young, you have highlighted the background into which the robot will enter: If it is designed and presented as a unique companion for the user as opposed to a mechanical tool like a vacuum cleaner, the response will be quite different. 5
Let’s talk about the anthropomorphization of robots—of seeing them as humanoid-like creatures. Anthropomorphism has been a catch phrase in my community which started with the idea that if you make machines more humanoid, then you can leverage gestures and gaze to support interactions. Over time, the anthropomorphism goes a lot deeper. When the gender factor comes in, this becomes really interesting and builds on the idea I mentioned earlier that people apply social norms and expectations to machines. There are 2 points I’d like to note. One is that while technologists often think that they’re making robots or machines gender neutral, I would argue that that is probably impossible. This is because people are gendered (whether it is binary, fluid, etc) and we bring that perspective to the table to understand our world. We should expect people to apply their existing stereotypes and assumptions surrounding gender to machines. When people anthropomorphize to even the smallest degree, we can expect them to start to apply their existing social norms and stereotypes, including ideas surrounding gender. For example, in some of our work using robots, we’ve tried to be gender neutral, such as giving the robot a neutral name, voice, and appearance. However, people still seem to find a way to explain it as male or female. The problem is that we put ourselves in these academic boxes with blinders on—such as focusing on algorithms or an application—and we fail to consider how gender does play into how people will interact with the system.
The second point I’d like to make is to reinforce a comment Dr Nomura made earlier: the danger for these robots to reinforce existing gender stereotypes—and this may be inadvertent if the technologists simply are not considering or educated on gender issues. One could easily imagine a robot being designed with traditional roles in mind, such as “male” robots doing heavy lifting and similar work, whereas housekeeping and personally interactive robots could be more feminine. If a nurse robot is designed to be more female, perhaps to leverage calming and supportive stereotypes, what message would this give to a young male who would like to go into health care? That only women are nurses?
There is a paper that I think is particularly influential by Eyssel and Hegel. 6 They demonstrated that just changing the hair style of a robot to be classically male or female was enough to encourage people to believe it was more appropriate for certain jobs and tasks, confined within traditional gender bounds. This is fascinating. In my own work, we’ve also looked at what people feel robots should do and analyzed what women and men think in general, instead of considering a male or female robot. In our work, we found that men are more likely to have strong opinions about what’s right and wrong broadly for society, while women are more likely to talk about their own lives and circles, especially about what’s safe in the home. Men are more likely to comment about military implications of robots broadly, while women are more likely to talk about specific impact on loved ones in the military. Another finding is that our male participants are a lot more enthusiastic about how robots can help them, particularly with domestic chores, whereas female participants were a lot less positive. The simple interpretation is that as men may like robots more than women, but there is a bigger context at play here in that the engineering and mechanical fields are severely male dominated. In North America, 20% of the computer science and engineering workforce is female, and while some top schools may approach 40% female reenrollment, most schools are at 20% or lower. 7 And so, the bigger issue is that women may simply be less interested in robots and potential applications because for so long they have been designed by and for men, for male-typical interests. This highlights a huge loss of potential impact for these technologies. It is crucial that we get more women into roles where they are designing and creating these technologies to ensure impact and potential more broadly for society.
There was an article recently in the New York Times about the new industries springing up among the relatively uneducated population in China for use in developing AI that had to do with the accurate identification of objects. There is a whole cadre of young people all over China making substantial amounts of money on labeling objects and feeding those labels back into the AI vehicle to correct and focus perceptions and ensure the accuracy of machines—for example, in self-driving cars. Apparently, the economics and the labor markets are profoundly influenced by the rising development of AI in very different ways, and as robots become more numerous and more employed in industrial and personal lives, so will employment opportunities increase.
I want to contribute to the discussion about robots. To put it in context, I was in a panel in France in April with thought leaders on AI in human health and the use of computers in health care. One speaker had a very interesting classification of AI: narrow AI, broad AI, and general AI. Narrow AI is already present: It includes facial recognition, with applications at bank ATMs and airport security checks. However, problems include reported errors in recognizing non-Caucasian skin tones, but still we do use facial recognition. Narrow AI is just a tool, as is voice assistance like Amazon’s Alexa. Alexa is not just answering consumer requests but is also used as a partner with the National Health Service in the United Kingdom to educate patients on health care. However, there are reports that Alexa may not accurately recognize drug names and may not function well when used by people with regional accents. At a higher technology level, broad AI is used in research and may include predictive analytics and prescriptive analytics. The transition to broad AI is more complex. An example is natural language processing for doctors’ notes: A challenge is the difficulty of some doctors to read their own notes. The third level of AI is what is called general AI. This perspective includes the humanoid robots portrayed in science fiction. This will probably not happen in the foreseeable future.
Although we don’t have anyone from NASA on our panel, there is a tremendous amount of discussion of the use of AI in warfare. A subset of that is the development of intelligent robots; this is not necessarily science fiction but as a continuation of robot technology changing from input–output to modeling on neural systems (characterized as “fuzzy”), which are plastic and adaptable and can make decisions. Some experts predict that by 2050, we will have autonomous robots that make decisions and generate more intelligent machines using their own resources.
I was thinking about killer robots and how long it will be before NASA or other international aerospace agencies develop more sophisticated robots than the ones currently being sent for exploration of other planets that will mimic some relevant human behavior. For instance, I can envision a human-like colony of robots that could build habitable infrastructures for humans, say on Mars. But we don’t have to consider the use of such robots only for activities outside of planet Earth. As we know, the formation of the Earth was an incredibly lengthy and complex process; the Earth has different layers. The region below the Earth’s surface is quite inhospitable to humans. But robots could be sent to investigate the phenomena that exist there and give us information about the formation of our planet and about any living creatures that exist below the surface. We could learn not only about the evolution of life on the planet but also find new sources of energy or precious metals. Artificial intelligence can allow us to explore the interior of volcanos and map the shifting of tectonic plates to predict earthquakes. We might even gain information that would help us reverse climate change.
Dr young, what are your ideas about a super intelligent robot and the danger of its becoming autonomous and definitely superior to humans?
The idea that we will have an intelligent robot that surpasses us in more than simple day-to-day tasks is 30 years into the future if ever, in my opinion. There’s a joke in the community that if you are attacked by a robot, just shine a flashlight at it (it may no longer see or will become confused). This highlights just how fragile robots are and what their simple limitations are. However, there is a huge middle range before the development of highly intelligent machines. We are already seeing simple cases such as the chatbot with the telephone company to adjust my bill, and we will increasingly see more such targeted, simple AI instances. Much bigger than any danger of super-intelligent robots being superior to humans is simple AI devices being manipulative and coercive. For me, there’s a real struggle between the people who develop the robot and their motivations and the people who use it, given that users may not understand what techniques or strategies the AI agent may be using. For example, in Canada, robots are starting to show up at airports and in banks, perhaps to be very engaging; the danger is that you have this little robot that is actually not very smart but has a face, arms, talks, and is friendly. As I mentioned, people will anthropomorphize this and interact with it in some ways as if it is alive. If you connect such a robot to a company with a large database and deep learning that knows the user’s history, finances, recent purchases, where they’ve been that day (eg, Google), then the robot can use human-like social techniques to influence behavior. This can be very powerful. Imagine a company putting this little machine in front of you at a bank to talk about your loan application. Can it emotionally manipulate you? It’s not difficult to consider a socially engineered situation. To the person, it does not need to be real “general AI” to have an impact; it will be sufficient simply to provide a life-like situation. There’s an important gender consideration here as well, as research has shown. Men and women may respond differently to male or female robots.
Dr Young, you commented in one of your papers about some control and safety issues but also the fascinating suggestion that people may fall in love with their robots and consequently may feel jealousy if a partner will decide also to use a shared robot for sex, for example.
I am very fortunate to have a colleague here at the University of Manitoba, Dr Neil MacArthur, with whom I have had several chats. He is a philosopher who specializes in sex bots. 8 Human brains are wired to apply existing social norms to these machines, and we easily fall into the trap of emotional attachment to the smiling, happy robot. It is perfectly natural, especially if you’re having sexual or intimate moments with it to have feelings related to love and attachment, even though some may be horrified by the idea. If loneliness is involved, a robot might seem a natural way to address that loneliness. My colleague highlighted that in the last 30 years, development of technological alternatives to sex (apart from pornography) has primarily been successful for female consumers, not males. I would be very curious to see if this changes. Again, given that the robotics industry is dominated by men and yet sex robots would be used by both sexes, we need to consider the limitations and technologic trajectories being promoted by this workforce.
There was a study very early on at MIT, where a whole group of young female postdocs were studying the interactions of people with robots. The director of the program actually became concerned because the women began to say that they actually preferred the robots, which were responsive, appeared to be thoughtful, and had a humanoid expression. It was not a joke. These women felt they could turn the robot off and put it away when it was not essential. When they wanted communication of a humanoid nature, these robots became more and more appropriate for interaction and were actually preferable to human partners.
We cannot deny that people often prefer robots for sexual use.
I want to highlight the use of sex robots and also of robots used by adults for loneliness. For me, the ethical key in all these situations is the balance between education and deception. For example, autistic children using robots might think the robot is actually alive: Is this okay? For companion robots, if you have a person who knows what the machine is offering—such as being a companion to watch a favorite comedy or for sex—I think that’s fine. The danger comes when people don’t understand how the technology works or what motives are behind the technology’s design for interaction and perhaps believe the machine is actually really alive. Companies are already marketing robotic and AI abilities using misleading buzzwords, and most people are educated in the technology enough to understand its capabilities.
But Dr Young, it is inevitable that these autistic children will personalize or anthropomorphize these robots. In the case of the patient with dementia, one of those most in need of companionship, human attendance falls off and such patients definitely personalize these entities as human. I think it is a very natural consequence of interaction with these machines, especially as you all point out, there is so much loneliness in the world.
The degree of attachment to and view of the robot varies enormously because of the heterogeneity of people’s needs and perceptions. Some individuals don’t need a robot to feel psychological or sexual attachment to an object. In fact, the adult industry provides “toys” and “dolls” for people’s use to which they develop a psychological or sexual attachment. There is a famous case of the Austrian Painter Oskar Kokoschka who had a doll modeled on his ex-mistress, Alma Mahler, the widow of the famous composer Gustav Mahler. Pathological as this may seem to some, I am sure this is not by any means unusual behavior judging by the success of the adult industry in this area. I have read recently and seen on YouTube some very interesting examples of “sexual robots” (male and female) using both traditional synthetic material and AI to take this type of fetishism to the next level.
I’d like to go back to something that you both touched on which is the question of data quality which applies to any device including robots. The machine is only as good as the data fed into it.
There are 2 types of data quality: problems with Big Data but also problems with small data (data that come from patients). When it comes to communicating the efficacy and safety of any device, we must consider the heterogeneity of people. Their attitudes toward AI are totally segment-specific. As we said earlier, at the point of care, the physician may have at his disposal a virtual, AI-aided scribe. The Millennial segment tends to be favorable to that because they are used to technology in their health-care searches. On the other hand, baby boomers with comorbidities do Internet searches on health just as often as Millennials, but often on different media, that is, desktops or tablets, whereas Millennials are almost entirely attached to their mobile phones. The younger group, therefore, may favor a virtual scribe because of Millennial familiarity with what could be called “Dr Google,” whereas for the older group, an adoption driver could be the doctor/patient face-to-face relationship.
The same principles of segmentation apply to physicians. Their attitudes tend to be specific to specialty. Radiologists, who are not dependent on face-to-face contact, tend to feel that AI makes their job easier but only as a tool and not as a diagnostic substitute. Pathologists, interestingly, are slower to adopt AI, for safety reasons. Many prefer the traditional whole-slide approach. 1 In ophthalmology, the Food and Drug Administration (FDA) granted approval to IDx, an AI diagnostics company, for the diagnosis of diabetic retinopathy. In oncology, there is more reluctance, because in the physicians’ view, especially in a first interview with a patient, when a diagnosis of cancer is given, a device could be perceived by the patient as disruptive.
Can any of you envision the creation of a robot who will sit at a desk and given in a first interview a simulation of empathetic and attentive response to the patient? I can and I think that’s coming.
I think it is definitely likely in terms of what is being sold and what is being marketed. In my university, many laboratories have robots in their work space, so I have had the opportunity to observe people interacting with very intelligent-looking or seemingly intelligent robots on a regular basis. I think we’re already seeing this distinction between what’s being sold in marketing materials and the media. If you go through the websites on robots or watch the news, they look very good. In fact, they are terrible in actual performance, and we see this on a day-to-day basis. I think that’s something we’re going to have to deal with in dealing with people selling these robots. It is easy to oversell capabilities, even to highly educated people. In private clinical practice, if there is resistance to having a patient see a robot, this is a very positive thing. The real quality of the interaction and the observational abilities of the robot are far from what people would expect it to be.
I agree with James in what he said about the algorithm itself being biased. We mentioned it when we talked about face recognition and the fact that skin colors may be source of error. That could also conceivably apply to voice recognition. What happens with an accent, for instance? That is starting to be an error in the real world. As we said earlier, this relates to the use of Amazon’s Alexa which is programmed in the United Kingdom to help patients get health information. If Alexa cannot recognize drug names, what kind of information are patients going to use? There is also the question of inequities in face and voice recognition. With voice recognition, is the person going to be articulate, is he going to have an accent and be able to express themselves in a way that a virtual voice assistant is going to recognize? In my view it is doubtful.
There is a whole school of thought that as we model robots to more mirror brain function and emulate neural connectivity and plasticity that we will be able to produce much more intelligent robots. This school of thought particularly warns that governance and control are very important elements as we develop these machines. Doctor Young, as you point out, we’re well behind in the capacity of these robots to respond except within very fixed parameters. But don’t you think there’s room for more development, particularly as the neural modeling of these robots improves?
I think it’s very far away. The more I deal with computer modeling, the more amazing I think humans are.
I don’t think a physician would agree to the acceptance of a robot as a peer diagnostician. What we have been discussing generally has a product-centric perspective. If we move from a product-centric perspective to a market-centric perspective, then the discussion becomes very different. Acceptability of AI in the real world really depends on the sophistication of the devices. In other words, narrow AI is moving from consumer use, such as facial recognition in bank ATMs, to medical applications such as robotic surgery. The latter is being improved in sophistication and control, but in some cases, such as specific tumors, surgeons may still see the traditional approach as safer.
For AI-aided diagnosis in clinical practice, the results are mixed. For example, a study of 37 000 computed tomography scans showed that the AI algorithm could interpret the scans 150 times faster than human experts, but its accuracy became poorer, given ample time for the human experts: What was gained in speed was lost in accuracy, if the time factor was not taken into account. 1
A related topic is that of GE Healthcare’s first FDA-approved AI design to detect pneumothorax. According to its developer, the algorithm was 96% accurate. However, another study showed that a procedure to diagnose pneumonia had an accuracy problem when used to process data from multiple hospitals. The machine was only as good as the data that fueled it. This shows the difference between an isolated laboratory environment and a real-world context.
There is a disconnect between what we’re seeing in the lab and in real-world applications. Scientists like to work in a lab in a very controlled environment; the numbers don’t necessarily work as well in the real world. It is important to communicate the accuracy and pitfalls of the program to the people using it. It would be terrible if a doctor really believed a device was more accurate than a specialist, when in fact it wasn’t. Artificial intelligence isn’t perfect; it is just a machine. We want to make sure the people using it, understand that, and that the data are presented in a way that calculates for uncertainty to avoid over-trust.
In actual medical practice, which I do, the patients harvest information which may not be critically evaluated, and its sources are not clearly defined. The skepticism of the patient about my opinion is a very challenging phenomenon with which I have to deal on a daily basis. The heterogeneity of the population is enormous, and I think that educating them about the limitations of artificial devices and AI in general is going to be a huge issue. Maybe it is an insurmountable issue.
I wanted to mention the psychological factor that enters into our relationship with the machine and also our relationship with the physician. As you have said, Dr Legato, there is such a heterogeneity in the human population. So, there will be those who will develop an abnormal psychological attachment to other people or to objects. Then as there are people who go to the clinic only to see their doctor to feel comforted by him or her, there will be others who will feel similar feelings and attachment to a robot. Human behavior can take all forms!
Exactly. And that brings us to one question which will come up—the question of privacy and security. The most important patient concern in recent surveys was privacy, as it relates to worldwide episodes of hacking. I’d like to point out by the way that there is a high incentive for hackers to get medical data. These are much more important and valuable to hackers than financial data. Why? Because with financial data, the bank would know right away. If you are a victim of a breach of health data, you may not know it, and your insurer may not know it for several months because it takes the insurers that long a time to assess your claims, and so on. The other point is that there is a difference in transparency between data in Europe and the United States. The European General Data Protection Regulation (EGDPR) gives the right to a citizen not to be subjected to a decision based solely on automated processing. The EGDPR also applies to all American companies because all American companies have members and users who are European citizens and therefore also affected. If there is a breach of EGDPR, the fine may be enormous, reaching as much as 4% of annual revenue.
The most intense concern about privacy is among Millennials. Surveys show them as the least trusting of consumer segments. This also applies to even younger generations. Among consumers, for every innovation we are talking about, there is a countertrend, especially in technology.
But it is so difficult to ensure that governments can agree on regulations. For example, concerning nuclear fission, synthetic biology, and of course AI and robots: How they will be used in peacetime; in warfare? I think the control of the development and deployment of these items is very difficult. I have a dim view, given our history, of regulating in a communally acceptable way all of these powerful new phenomena.
I agree and again, going back to the question of trend/countertrend I think we should talk about the difference between innovation developed in an isolated, highly controlled environment which is the lab and what happens in the real world—in particular, in the world of practice and clinical care.
The encounter between patient and physician is characterized by a dynamic of shared decision-making. This is a key driver of adherence. If the patient doesn’t believe in the conversation that he or she has had with the doctor, he or she is not going to comply. Is there going to be acceptance of the fact that after taking the patient’s history, the decision about diagnosis and treatment is in part dictated by an algorithm in the computer? Again, this could be very segment-specific. Even if a therapeutic decision is accepted, will it lead to better outcome and adherence.
Artificial intelligence–assisted coaching is no longer in the lab. We have virtual devices such as Livongo, which have a combination of possibly chatbot conversation and possibly also depending on the tenor of the conversation, referrals to other health-care interventions. We are also seeing the use of wearables such as the Apple Watch or Fitbit to monitor activity. Here again, there is trend and countertrend. The countertrend is that there have been a number of studies that show a very high attrition of the use of these devices/wearables. It is called “digital fatigue.” People have over 250 000 health apps available on wearables. Most people use maybe 10 of them. After a few months, they get busy and they stop using the device. The question again is the balance between the availability of the coaching and patient adherence.
The most resistant and skeptical population tends to be the Millennials; they are the ones that put ad blockers on their cell phones. Several years ago, Coca-Cola introduced New Coke. They did taste tests and redesigned their beverage, but this led to a market failure. The reason is that New Coke was launched while the company was withdrawing the initial version. The wrong way to introduce this technology is to tell consumers: “you don’t have a choice.” This type of control is very powerful and has to be taken into account.
So maybe we have to draw a line between what AI is capable of doing and what it is that we can actually achieve. Who will regulate the application of AI? And what will be the market for AI? These considerations will involve all areas of AI technology.
I am afraid we are out of time. Thank-you all for you insights, observations, and valuable commentary.
Footnotes
Authors’ Note
Our institution does not require ethical approval for reporting individual cases or case series. Informed consent for patient information to be published in this article was not obtained because no patient information was included.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.