Digital Exhaust or Digital Gold? The Real Value of AI-Generated Clinical Visit Transcripts

How Are AI Scribes Changing Clinical Visits?

AI scribes are software systems that listen to consultation-room conversations, transcribe them, and summarize them into draft clinical reports . Organizations are adopting them rapidly to reduce physicians' documentation burden and burnout risk. In some health systems, they are now used in millions of visits each year .

An important detail: because most of these systems technically do not qualify as “medical devices” , they do not undergo conventional FDA review. Despite widespread clinical use, independent, systematic assessment of safety and accuracy is therefore often lacking.

Each visit first produces an audio recording and a complete transcript , from which a summary note is generated. The physician checks, edits, and signs it, and the note enters the patient's record. The critical question is: What happens to the transcript and audio recording afterward?

Digital Gold, Not Digital Exhaust: What Do Transcripts Actually Offer?

To understand an AI scribe's real performance, we need two things:

• The original dialogue (transcript),
• The summary note generated from it.

But after transcripts are deleted, only flattened summary notes remain. We then cannot objectively measure:

• What the AI captured correctly,
• Where it made omissions, distortions, or hallucinations ,
• How often the physician noticed and corrected these errors

.

The Patient's Own Words: A New Window Into Diagnosis, Communication, and Medical History

Transcripts offer a unique resource not only for AI evaluation but also for clinical research . For example:

• How often physicians interrupt patientsand whether this is linked to diagnostic errors,
• Which subtle but critical symptoms patients with rare cancers described years before diagnosis ,
• What words patients use to describe symptoms outside medical terminology

were previously questions almost impossible to study with large-scale data. Properly retained, anonymized transcripts now allow researchers to investigate them across thousands to millions of visits .

These analyses offer a unique opportunity to understand how not only AI tools but our own clinical communication styles relate to patient outcomes.

What Steps Could Support Safe Transcript Retention?

1. Advocacy by the Clinical and Academic Community

In most organizations, risk-management teams and lawyers decide whether transcripts are retained. Researchers and clinicians should clearly emphasize in internal discussions:

• AI scribes have not yet been independently and systematically evaluated,
• Without transcripts, these tools cannot generate robust evidence about patient safety,

. Moreover, as patients increasingly record consultations on their own phones , many alternative recordings will already exist.

2. Sharing Policies Across Organizations

No organization wants to be “the first to act alone.” Anonymous surveys and networks can share:

• How many organizations retain transcripts,
• Under what conditions and for how long,
• Good practices for patient consent, anonymization, and access permissions

. Each organization can then decide within a collective frame of reference, rather than in isolation .

3. Graduated, Secure Retention Models

Instead of “keep everything or delete everything,” transcripts can be managed using a tiered risk model :

• Irreversible anonymization: Transcripts masked so that neither patient nor physician can be reidentified have considerable research value.
• Use solely for quality improvement: In some legal systems, data retained with restricted access may receive greater protection in litigation.
• For broad research use: For data linkable to patients or physicians, new mechanisms may be needed: legal “safe harbors” and periods of malpractice blackout during which transcripts are protected from discovery for a defined time.

Such legal changes will not happen automatically; active advocacy by researchers and clinicians will be essential to making progress.

Conclusion: Preserving the Dialogue, Not Just the Note

AI scribes have been adopted by administrators and frontline physicians with unusual speed. But by systematically destroying the raw data they produce, we also undermine medicine's ability to evaluate and improve itself.

Clinical conversations are more than raw material for a note; they are unique records for patient safety, diagnostic accuracy, communication quality, and AI reliability . Researchers, clinicians, and legal experts can work together to design models that preserve transcripts securely and ethically . Otherwise, data viewed as a “risk” by health systems will remain digital gold only for outside companies.

Reference

Goodman, K. E., & Morgan, D. J. (2025). Digital Exhaust or Digital Gold? The Value of AI-Generated Clinical Visit Transcripts. New England Journal of Medicine. Published: 3 December 2025. DOI: 10.1056/NEJMp2514616. The full article is available on the NEJM website .