AraHealthQA 2025 Shared Task

Bridging the gap in Arabic medical question answering for Large Language Models (LLMs). Part of #ArabicNLP2025, co-located with #EMNLP2025.

Read the Paper

Track 1: MentalQA 2025 (Mental Health)

This track focuses on the mental health domain, addressing the urgent need for culturally and linguistically tailored resources in Arabic. It leverages the newly constructed **MentalQA dataset**.

Sub-Task 1: Question Categorization

Classify patient questions into categories like diagnosis, treatment, etc. (Multi-label classification).

Evaluation: Weighted F1, Jaccard Score

Competition Dataset

Sub-Task 2: Answer Categorization

Classify responses based on strategy (e.g., emotional support, information). (Multi-label classification).

Evaluation: Weighted F1, Jaccard Score

Competition Dataset

Sub-Task 3: Question Answering

Generate appropriate mental health answers based on the question and context. (Text generation).

Evaluation: BERTScore

Competition Dataset

Track 2: MedArabiQ 2025 (General Health)

This track focuses on general Arabic health QA, based on the **MedArabiQ dataset**. It aims to catalyze progress in Arabic medical NLP by creating new benchmarks.

Sub-Task 1: Multiple Choice QA

Choose the correct answer from a fixed set of choices for Arabic medical questions. (Classification).

Evaluation: Accuracy

Competition

Datasets:

1. Multiple Choice Questions 2. Multiple Choice Questions with Bias 3. Fill-in-the-blank with Choices

Sub-Task 2: Open-Ended QA

Generate correct answers for medical fill-in-the-blank and open-ended clinical questions. (Generation).

Evaluation: BLEU, ROUGE, BERTScore

Competition

Datasets:

1. Fill-in-the-blank without Choices 2. Patient-Doctor Q&A 3. Q&A with GEC 4. Q&A with LLM Modifications