Job Description
On behalf of our client, a leading technology organization based in Rusayl – Oman, JM International SPC is seeking a highly skilled AI Quality Engineer to support a national Large Language Model (LLM) initiative focused on delivering high-quality, reliable, and culturally accurate Arabic AI solutions for government use.
This role is critical to ensuring robust quality frameworks across data, models, and RAG systems prior to large-scale deployment.
Ideal Profile
• Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, Machine Learning, or a related field
• 4+ years of experience in AI/ML quality assurance, with strong focus on NLP or LLM systems
• Solid understanding of LLM evaluation methodologies and quality benchmarking
• Experience working with Arabic language datasets and NLP challenges
• Proficiency in Python and AI/ML testing or evaluation frameworks
Apply now and contribute to a nationally significant AI initiative shaping the future of Arabic language technology.
Responsibilities
Key Focus
• Design and implement data quality frameworks for Arabic LLM training and evaluation
• Establish quality gates across data preparation, model training, evaluation, and RAG pipelines
• Define acceptance criteria and quality metrics for AI system sign-off
• Implement automated testing, monitoring, and regression frameworks
• Ensure cultural accuracy, bias detection, and fairness in model outputs
• Support benchmarking, documentation, and quality governance processes