ChatGPT-Assisted Answers to Patient Questions – Nurses Are Positive, Doctors Skeptical

The Role of AI in Healthcare

ChatGPT has brought the AI revolution into healthcare. Patient portals used for patient–clinician communication are undergoing a major transformation with AI integration. AI-powered chatbots that generate automatic replies to portal messages promise to reduce healthcare workers' workloads and speed up processes. However, not all professional groups welcome this innovation in the same way. On October 14, 2024, JAMA Network Open published a study highlighting differences in attitudes between nurses and other healthcare professionals. In particular, the study found that nurses were more accepting of this technology . What place will AI-assisted replies have in healthcare?

Why Do Nurses Support AI-Assisted Replies?

The study examined a chatbot based on the large language model (LLM) GPT-4, tested in the University of Colorado Health system from September 2023 to March 2024. Used in 9 clinics (6 primary care and 3 specialty clinics), it generated 21323 draft replies to patient portal messages. Healthcare workers used 12% of these replies (2596). After two weeks of use, feedback was collected from 69 healthcare professionals: 12 nurses, 14 resident physicians and 43 specialists.

Nurses embraced the benefits of chatbot use more than other healthcare workers. Net promoter score (NPS) data showed a recommendation score of 58 among nurses, compared with -29 among residents and -43 among clinicians (P=0.004). These results suggest that nurses view the technology as an effective tool for reducing workload.

The Benefits of AI-Assisted Replies in the Data

Several important factors explain why nurses viewed this technology more positively than other healthcare workers. The study found that nurses reported benefits from ChatGPT in the following areas:

• Responding to messages more quickly: 92% of nurses, 50% of residents and 46% of specialists said the chatbot helped them respond to messages faster (P=0.03).
• Expressing empathy more effectively: 92% of nurses, 43% of residents and 42% of specialists said the chatbot made it easier to produce empathetic replies (P=0.01).
• Low risk of providing incorrect information: 67% of nurses, 14% of residents and 28% of specialists said the risk of the chatbot providing incorrect information was minimal (P=0.04).

75% of nurses said the chatbot reduced the need to forward messages to doctors (P=0.002). This is an important finding suggesting that it reduced nurses' communication burden with doctors and gave them greater independence in their work.

Concerns Among Other Healthcare Workers

Residents and clinicians were less positive about chatbot use than nurses. Residents in particular struggled to adopt it because messages containing clinical information fell outside its scope. Specialists received more complex messages, which GPT found harder to handle effectively.

The researchers stress that this reflects differences in professional responsibilities and that the chatbot should be tailored to each group. Nurses generally answer more basic patient questions, while doctors may face complex medical problems. AI-assisted replies therefore need to be adapted differently for each healthcare professional group.

The Future of AI: What Steps Should Be Taken?

This study suggests that large language models can be useful in healthcare but that their use requires better optimization. Future systems that recognize differences between roles and generate replies suited to each professional group may more effectively reduce workloads for nurses, residents and specialists.

Although AI-assisted replies may speed up healthcare, complex medical situations still require human oversight and assessment. This research shows much greater acceptance among certain groups, such as nurses, offering important insight into where AI-assisted health technologies may contribute most.

Conclusion: Are AI-Assisted Replies Here to Stay in Healthcare?

AI-assisted patient portal messages can reduce healthcare workers' workloads, but the technology must be adapted to each professional group. Nurses' positive views suggest success in certain areas, while complex medical questions still require a human touch. Considering differences between roles is essential for wider future use of AI in healthcare.