ASCAT: An Arabic Scientific Corpus and Benchmark for Advanced Translation Evaluation

arXiv cs.CL / 4/3/2026

💬 OpinionIdeas & Deep AnalysisTools & Practical UsageModels & Research

共有:

Key Points

The paper introduces ASCAT, a high-quality English–Arabic parallel corpus and benchmark specifically built for evaluating scientific translation of full abstracts rather than short or single-domain sentences.
ASCAT is constructed via a systematic multi-engine translation pipeline using generative AI (Gemini), transformer-based models (Hugging Face quickmt-en-ar), and commercial MT APIs (Google Translate, DeepL), followed by human expert validation across lexical, syntactic, and semantic levels.
The benchmark covers five scientific domains—physics, mathematics, computer science, quantum mechanics, and artificial intelligence—and each abstract averages about 141.7 English words and 111.78 Arabic words.
The released corpus statistics include 67,293 English tokens and 60,026 Arabic tokens with an Arabic vocabulary of 17,604 unique words, reflecting Arabic’s morphological richness.
When evaluated on three state-of-the-art LLMs (GPT-4o-mini, Gemini-3.0-Flash-Preview, and Qwen3-235B-A22B), ASCAT yields BLEU scores of 37.07, 30.44, and 23.68 respectively, demonstrating its discriminative value for scientific MT evaluation and domain model training.

Abstract

We present ASCAT (Arabic Scientific Corpus for Advanced Translation), a high-quality English-Arabic parallel benchmark corpus designed for scientific translation evaluation constructed through a systematic multi-engine translation and human validation pipeline. Unlike existing Arabic-English corpora that rely on short sentences or single-domain text, ASCAT targets full scientific abstracts averaging 141.7 words (English) and 111.78 words (Arabic), drawn from five scientific domains: physics, mathematics, computer science, quantum mechanics, and artificial intelligence. Each abstract was translated using three complementary architectures generative AI (Gemini), transformer-based models (Hugging Face \texttt{quickmt-en-ar}), and commercial MT APIs (Google Translate, DeepL) and subsequently validated by domain experts at the lexical, syntactic, and semantic levels. The resulting corpus contains 67,293 English tokens and 60,026 Arabic tokens, with an Arabic vocabulary of 17,604 unique words reflecting the morphological richness of the language. We benchmark three state-of-the-art LLMs on the corpus GPT-4o-mini (BLEU: 37.07), Gemini-3.0-Flash-Preview (BLEU: 30.44), and Qwen3-235B-A22B (BLEU: 23.68) demonstrating its discriminative power as an evaluation benchmark. ASCAT addresses a critical gap in scientific MT resources for Arabic and is designed to support rigorous evaluation of scientific translation quality and training of domain-specific translation models.