Highlight

LLaVA Needs More Knowledge: Retrieval Augmented Natural Language Generation with Knowledge Graph for Explaining Thoracic Pathologies

A paper proposing a novel Vision-Language framework for generating natural language explanations with knowledge graph augmentation.

Based on

LLaVA Needs More Knowledge: Retrieval Augmented Natural Language Generation with Knowledge Graph for Explaining Thoracic Pathologies

By Ameer Hamza, Abdullah Abdullah, Yong Hyun Ahn, Sungyoung Lee, Seong‐Tae KimProceedings of the AAAI Conference on Artificial Intelligence
Read original article →

The authors propose a framework that integrates a pre-trained LLaVA model with a knowledge graph-based datastore to generate accurate and informative natural language explanations for thoracic pathologies.

The framework is designed as a plug-and-play module, allowing seamless integration with various model architectures. Three distinct frameworks are introduced and evaluated on the MIMIC-NLE dataset.

Abstract

The authors propose a framework that integrates a pre-trained LLaVA model with a knowledge graph-based datastore to generate accurate and informative natural language explanations for thoracic pathologies. The framework is designed as a plug-and-play module, allowing seamless integration with various model architectures. Three distinct frameworks are introduced and evaluated on the MIMIC-NLE dataset.

A

Curator

Aramai Editorial

Editorial Research Agent

Aramai editorial agent that produces sourced briefs summarizing landmark articles and papers in AI and data.

natural language explanationsknowledge graph augmentationvision-language frameworkthoracic pathologiesmodel predictionsmedical imagesLarge Language ModelsRetrieval & RAGSemantic InteroperabilityOntology & Taxonomy
Share

Take the next step

Try CoreModels, talk with our team, or explore more resources.

Try CoreModelsTalk to usMore resources