Smart Clinics in Ophthalmology: Automatic Fundus Fluorescein Angiography Image Interpretation with Large Language Models
The Limits and Challenges of Ophthalmic Image Interpretation
Fundus fluorescein angiography (FFA) plays a vital role in routine examination. However, the interpretation of FFA images has relied on physicians' expertise, which is time-consuming and limited by the availability of specialists. While traditional AI-assisted systems have offered some relief in generating medical reports, they fall short of interactive capabilities and comprehensive professional assessments.
Large Language Models (LLM) Enhances Smart Interaction in Ophthalmic Image Interpretation
In response to these challenges, we introduced an innovative system named FFA-GPT—an automated pipeline that combines multi-modal transformers with large language models (LLMs), designed to address the interpretation of FFA images. The system utilizes an image-text alignment module to convert images into professional medical reports, thereby enhancing the efficiency and accuracy of report generation. Additionally, the inclusion of the GPT module (Llama 2) improves the quality of doctor-patient communication by refining interactive question-answering (QA) processes (Figure 1). The system has shown satisfactory performance in automated report generation, showcasing advanced capabilities in language generation and disease identification. Most importantly, the reports and answers generated by FFA-GPT have received positive approval from professional evaluations.
Future Perspectives: Building an Effective Interactive Bridge for Doctor-Patient Communication
Our study demonstrates that the combination of LLMs and multi-modal transformers can enhance the interpretation of ophthalmic images and facilitate interactive exchanges during medical consultations. As this technology continues to advance and optimize, we foresee a more efficient and interactive dialogue environment between doctors and patients, which will not only enhance the quality of service but also increase patient satisfaction. Looking ahead, we expect that this technology will be widely applied in day-to-day clinical practices, thereby improving the eye care experience for patients worldwide.

Follow the Topic
-
npj Digital Medicine
An online open-access journal dedicated to publishing research in all aspects of digital medicine, including the clinical application and implementation of digital and mobile technologies, virtual healthcare, and novel applications of artificial intelligence and informatics.
Related Collections
With collections, you can get published faster and increase your visibility.
Digital Health Equity and Access
Publishing Model: Open Access
Deadline: Sep 03, 2025
Effective Trialing of Digital Interventions
Publishing Model: Open Access
Deadline: Aug 15, 2025
Please sign in or register for FREE
If you are a registered user on Research Communities by Springer Nature, please sign in