Arabic (Gulf) AI Evaluation Specialist

Who you are

You are a detail-oriented individual with a Bachelor's degree or equivalent experience in Linguistics, Computational Linguistics, Communications, Technical Writing, or a related analytical field. Your native-level fluency in Gulf Arabic allows you to understand the nuances of the language and culture, which is essential for evaluating AI systems effectively. You have experience in AI evaluation, prompt engineering, or linguistic QA, and you are familiar with the regional norms and high-context communication styles prevalent in the GCC region. You are eager to learn and adapt, as you will be required to attend webinars and continuous learning sessions to stay updated on best practices in AI evaluation.

What you'll do

In this role, you will be instrumental in refining and evaluating large language models (LLMs) by designing scenario-based and edge-case prompts to test AI behavior. You will develop evaluation rubrics to assess AI responses across various criteria, including instruction-following, factuality, tone, safety, refusals, and helpfulness. Your responsibilities will include performing side-by-side evaluations of AI outputs and scoring them on a defined scale. You will also create high-quality source documents that serve as the single source of truth for testing and write accurate Golden Responses that handle ambiguity effectively. Your expertise will contribute to building smarter, more reliable, and helpful AI technology.

What we offer

This position offers a competitive pay rate of $10 USD per hour, with a commitment of 40 hours a week, Monday to Friday. The project duration is three months, starting on February 2nd. You will have the opportunity to work remotely from Egypt, allowing for flexibility in your work environment. As part of the team, you will engage in continuous learning and development, enhancing your skills in AI evaluation and contributing to cutting-edge technology in the field.