Law Model Prompt Evaluator
Mercor • Remote • Posted 17 days ago
Education
Any
Type
Pay Rate
$87.5/task
Posted
17d ago
✅ Applying through this link gives you a verified candidate referral.
Referrals from verified candidates give your profile a visibility boost and help support our platform at no cost to you.
This position is hosted on an external talent platform. Please only apply for this position if it fits your skills and interests.
About this Role
Role Overview
We are seeking expert legal professionals to author and verify high-quality open-ended prompts for AI model evaluation. You will craft and review challenging, unambiguous legal problems across core subdomains, assessing AI reasoning quality and helping establish rigorous evaluation standards for frontier language models.
Role Overview
Authoring Task
Verification Task
Law Subdomains Covered
Key Responsibilities
Ideal Qualifications
More About the Opportunity
Role Overview We are seeking expert legal professionals to author and verify high-quality open-ended prompts for AI model evaluation. You will craft and review challenging, unambiguous legal problems across core subdomains, assessing AI reasoning quality and helping establish rigorous evaluation standards for frontier language models. You will be assigned one of two task types: Law Subdomains Covered Professional & Statutory Law, Jurisprudence & Legal Theory, International Law, Contract Law, Criminal Law, Constitutional Law, Regulatory & Administrative Law. Key Responsibilities Ideal Qualifications More About the Opportunity We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
- Authoring Task — Create 5 original, open-ended prompts from your assigned subdomain at varying difficulty levels (undergraduate, advanced undergraduate, or graduate/professional). Prompts should require human judgment to evaluate the quality of the AI's response, such as legal reasoning, statutory interpretation, or case analysis.
- Verification Task — Review 5 authored prompts for clarity, scope alignment, difficulty accuracy, and uniqueness. Edit prompts and difficulty ratings where needed.
- Author clear, unambiguous, open-ended legal prompts that elicit evaluable AI responses
- Verify prompts are within the scope of the assigned subdomain and correctly rated for difficulty
- Ensure all 5 prompts in a task are sufficiently distinct from one another with varying difficulty levels
- Apply expert judgment to assess the depth and quality of legal reasoning required
- Edit prompts and difficulty assignments where standards are not met
- JD, LLM, SJD, or Master's degree or higher in Law or a closely related field
- 2–6 years of professional experience in legal practice, academia, or policy
- Strong command of legal reasoning, statutory interpretation, and jurisprudential theory
- Bar admission, judicial clerkship, or legal research experience is a strong plus
- Excellent written English and ability to craft precise, well-scoped legal questions
- Expected commitment: 10+ hours/week
- Asynchronous, fully remote work
Requirements
- Must be eligible to work in Remote
- Fluent proficiency in English (Written & Verbal)
- Reliable high-speed internet connection
- Bachelor's degree or equivalent professional experience
- Demonstrated expertise in Writing
Eligible Languages
Fluent proficiency in English
Compensation Analysis
Work from anywhere, at any time. This fully remote position ($87.5/hr) breaks down geographic barriers, allowing you to earn US-competitive rates regardless of your local market. It is a perfect stepping stone for building a career in the data labeling and AI training ecosystem.
Skills & Categories
Explore other opportunities in related specializations:
Related Jobs
Browse All Jobs from Mercor
Discover more opportunities on Mercor that match your skills and interests.
View All Mercor Jobs →Community Reviews
Leave your review
Frequently Asked Questions
Is this for freelancers or full-time employees?
Both. Mercor tries to match you with clients who want long-term contractors. Unlike other platforms where you log in and grab small tasks, Mercor matches you with one company for a steady role (e.g., 'Python Tutor for 3 months').
I'm not comfortable on camera. Can I still apply?
No. The application requires a video interview with an AI avatar. The AI asks you questions about your resume, and the video is shared with potential clients to prove your communication skills.
Does it cost money to join?
No. You should never pay to join these platforms. Mercor makes money by charging the client a fee on top of your hourly rate.
What does the work actually look like?
It is practical, hands-on data work. You might be recording short videos, categorizing images, rating text responses, or analyzing data. The tasks are designed to be short and distinct—typically 5-60 minutes per task.
How flexible is the schedule?
Extremely. This is true "log in and work" flexibility. You can usually work for 20 minutes or 4 hours depending on your availability. There are rarely minimum hour requirements, making it ideal for side income.
Is there an interview?
Usually, no. Hiring for these roles is almost entirely based on passing an automated assessment or "qualification" task. If you pass the test, you get access to the work.
How soon will I start?
Important: Mercor is a talent marketplace, not a task queue. Applying puts you in a pool of candidates. You will only start working when a specific client (like a major AI lab) selects your profile. This matching process can take weeks.