Generalist - English & Arabic

Location: Geography restricted to Egypt, Saudi Arabia, UAE, USA Type: Full-time or Part-time Contract Work Fluent Language Skills Required: English & Arabic

Why This Role Exists

Location

Type

Fluent Language Skills Required

Why This Role Exists

improving general chat behavior

What You’ll Do

Who You Are

Bachelor’s degree

native speaker

ILR 5/primary fluency (C2 on the CEFR scale)

Arabic

significant experience using large language models

excellent writing skills

strong attention to detail

adaptable

structured analytical thinking

excellent college-level mathematics skills

Nice-to-Have Specialties

RLHF, model evaluation, or data annotation work

high-quality written content

fine-grained qualitative judgments

Familiarity with evaluation rubrics

What Success Looks Like

Why Join Mercor

Location: Geography restricted to Egypt, Saudi Arabia, UAE, USA Type: Full-time or Part-time Contract Work Fluent Language Skills Required: English & Arabic Why This Role Exists Mercor partners with leading AI teams to improve the quality, usefulness, and reliability of general-purpose conversational AI systems. These systems are used across a wide range of everyday and professional scenarios, and their effectiveness depends on how clearly, accurately, and helpfully they respond to real user questions. This project focuses on evaluating and improving general chat behavior in large language models (LLMs). You will assess model-generated responses across diverse topics, provide high-quality human feedback, and help ensure AI systems communicate in ways that are accurate, well-reasoned, and aligned with human expectations. What You’ll Do Evaluate LLM-generated responses on their ability to effectively answer user queries Conduct fact-checking using trusted public sources and external tools Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies Assess reasoning quality, clarity, tone, and completeness of responses Ensure model responses align with expected conversational behavior and system guidelines Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines Who You Are You hold a Bachelor’s degree You are a native speaker or have ILR 5/primary fluency (C2 on the CEFR scale) in Arabic You have significant experience using large language models (LLMs) and understand how and why people use them You have excellent writing skills and can clearly articulate nuanced feedback You have strong attention to detail and consistently notice subtle issues others may overlook You are adaptable and comfortable moving across topics, domains, and customer requirements You have a background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering) You have excellent college-level mathematics skills Nice-to-Have Specialties Prior experience with RLHF, model evaluation, or data annotation work Experience writing or editing high-quality written content Experience comparing multiple outputs and making fine-grained qualitative judgments Familiarity with evaluation rubrics, benchmarks, or quality scoring systems What Success Looks Like You identify factual inaccuracies, reasoning errors, and communication gaps in model responses You produce clear, consistent, and reproducible evaluation artifacts Your feedback leads to measurable improvements in response quality and user experience Mercor customers trust the quality of their AI systems because your evaluations surface issues before public release Why Join Mercor At Mercor, you’ll work at the frontier of human-in-the-loop AI development, directly shaping how advanced language models behave in the real world. This role offers flexible, remote contract work and the opportunity to contribute meaningfully to AI systems used by millions of people. Contract rates are competitive and aligned with the level of expertise required and scope of work. We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

• Evaluate LLM-generated responses on their ability to effectively answer user queries
• Conduct fact-checking using trusted public sources and external tools
• Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies
• Assess reasoning quality, clarity, tone, and completeness of responses
• Ensure model responses align with expected conversational behavior and system guidelines
• Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines
• You hold a Bachelor’s degree
• You are a native speaker or have ILR 5/primary fluency (C2 on the CEFR scale) in Arabic
• You have significant experience using large language models (LLMs) and understand how and why people use them
• You have excellent writing skills and can clearly articulate nuanced feedback
• You have strong attention to detail and consistently notice subtle issues others may overlook
• You are adaptable and comfortable moving across topics, domains, and customer requirements
• You have a background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering)
• You have excellent college-level mathematics skills
• Prior experience with RLHF, model evaluation, or data annotation work
• Experience writing or editing high-quality written content
• Experience comparing multiple outputs and making fine-grained qualitative judgments
• Familiarity with evaluation rubrics, benchmarks, or quality scoring systems
• You identify factual inaccuracies, reasoning errors, and communication gaps in model responses
• You produce clear, consistent, and reproducible evaluation artifacts
• Your feedback leads to measurable improvements in response quality and user experience
• Mercor customers trust the quality of their AI systems because your evaluations surface issues before public release