Caution: May Hallucinate When Prompted

There has been plenty of negative press for AI these days. After all, Anthropic’s employees just shared a warning that AI could lead to human extinction within the next decade.¹ As is the case with most AI conversations, the current risk of danger is low, but it has the potential to get out of hand quickly.

​But sometimes—when implemented carefully—AI really can impress us. For example, researchers at Duke created a deep-learning system that used OCT images to reliably predict when anti-VEGF treatment should be initiated for patients converting to wet AMD, while also measuring changes in outer retinal thickness.²

Most of us are already dabbling in AI for certain tasks, such as summarizing research, drafting emails, and finding good restaurants in New Orleans during AAO. Yet, we must remember that HIPAA reigns supreme, and using AI with abandon could put patient privacy and safety at risk. Researchers at Stanford and Harvard collaborated on the NOHARM trial, finding that recommendations from large-language models had the potential for severe harm in up to 22.2% of cases.³ There remains a gap between our interest in using AI to our advantage, and our ability to do so safely.

This issue is designed to start to fill that gap and help us better understand the potential of AI in our practices—and the pitfalls to avoid. For one thing, there are growing concerns that relying too much on AI in the clinic could be eroding clinical decision-making skills.⁴ In one of our featured articles, Andreas Pollreisz, MD, and colleagues address this issue (known as deskilling and never skilling) by drawing parallels between aviation and the field of retina. He notes that, just as flight automation still requires skilled pilots, automation in retina can augment, but not replace, the skill we bring to our profession.

In keeping with the pilot theme, Daniel Ting, MD, PhD, and Stanley Poh, FRCOphth, explore the future of AI in the OR as a digital copilot. While still theoretical, it may one day enhance surgical planning, intraoperative guidance, training, documentation, and postoperative evaluation.

As for clinical efficiencies, Fares Antaki, MDCM, introduces chatbots—what they are, what they can do—while Fatima Kalabi, MD, and her colleagues show us how, exactly, we can prompt them safely. Digging a little deeper, Prashant D. Tailor, MD, and his team summarize their recent research on AI’s utility for pre-charting, note-taking, and referral letters. For example, they found that 85% of non-ophthalmology clinicians preferred referral notes that included an AI-generated plain-language summary.

One of the most promising applications of AI is for clinical trial recruitment and data analysis. Sunil Gupta, MD, and Joshua Koo, MS, BS, share their research on an investigational AI model that can automatically flag eligible candidates, potentially improving enrollment rates.

AI has so much potential, and it’s already making our lives a little easier in many ways. If we are careful, AI can streamline much of our workflow, too. We just have to implement it with caution. 

1. Gerken T. Anthropic researcher believes more than 10% chance AI ‘could kill all humans’. BBC. September 9, 2026. Accessed September 11, 2026. www.bbc.com/news/articles/ckgwy1k42w4o

2. Gao Q, Kuo D, Amason J, et al. Modular multi-task deep learning framework for prediction of treatment initiation in neovascular age-related macular degeneration modular AI for NVAMD treatment initiation study-MANTIS. [published online ahead of print September 1, 2026). Br J Ophthalmol. doi.org/10.1136/bjo-2025-328327

3. Wu D, Haredasht FN, Maharaj SK, et al. First, do NOHARM: towards clinically safe large language models [published online ahead of print December 17, 2026]. ArXiv.

4. Bajaj S, Sakran J. What happens when medical students rely on AI – and never develop their own judgment? The Guardian. August 10, 2026. Accessed August 10, 2026. www.theguardian.com/commentisfree/2026/aug/10/ai-medical-students-judgment