Suzhou: THIQAH, in cooperation with King Abdullah University of Science and Technology (KAUST), presented a distinguished research paper as part of the Third Arabic Natural Language Processing Conference (ArabicNLP 2025), hosted in the Chinese city of Suzhou in conjunction with EMNLP 2025, one of the world’s leading conferences in artificial intelligence and language processing.
According to Saudi Press Agency, the paper, titled ‘ALARB: An Arabic Legal Argument Reasoning Benchmark’, focuses on developing the first Arabic-language benchmark designed to evaluate the reasoning capabilities of language models in legal contexts.
The study aims to bridge an existing gap, as the Arabic language lacks precise reference standards for processing legal texts compared with other languages. This makes the paper a pioneering contribution toward boosting the presence of the Arabic language in AI research.
During the conference, the research team from THIQAH and KAUST presented the methodology used in building the dataset, which was developed through collecting, organizing, and analyzing published legal cases. These were restructured according to four main components: facts, reasoning, judgment, and relevant regulations. This structure enables language models to be trained to perform multiple legal tasks with higher accuracy.
Initial tests of the benchmark on several global models – including GPT-4, Gemma, and Aya – showed significant improvements in model performance when processing Arabic legal texts, demonstrating the effectiveness of the ALARB benchmark in advancing Arabic-based evaluation standards capable of guiding future research in this field.