Law Model Prompt Evaluator
Mercor • Remote
Education
Bachelor's
Type
Hourly
Pay Rate
$75–$100/hr
Listed
175d ago
Apply opens Mercor in a new tab.
Apply Now → ⚡ Boost your chances - Optimize your resume with Rezi.aiWhat We Know About This Role
- Weekly hours
- 10 hrs/week
About this role
From the Mercor listing
Role Overview
We are seeking expert legal professionals to author and verify high-quality open-ended prompts for AI model evaluation. You will craft and review challenging, unambiguous legal problems across core subdomains, assessing AI reasoning quality and helping establish rigorous evaluation standards for frontier language models.
- Authoring Task — Create 5 original, open-ended prompts from your assigned subdomain at varying difficulty levels (undergraduate, advanced undergraduate, or graduate/professional). Prompts should require human judgment to evaluate the quality of the AI's response, such as legal reasoning, statutory interpretation, or case analysis.
- Verification Task — Review 5 authored prompts for clarity, scope alignment, difficulty accuracy, and uniqueness. Edit prompts and difficulty ratings where needed.
- Author clear, unambiguous, open-ended legal prompts that elicit evaluable AI responses
- Verify prompts are within the scope of the assigned subdomain and correctly rated for difficulty
- Ensure all 5 prompts in a task are sufficiently distinct from one another with varying difficulty levels
- Apply expert judgment to assess the depth and quality of legal reasoning required
- Edit prompts and difficulty assignments where standards are not met
- JD, LLM, SJD, or Master's degree or higher in Law or a closely related field
- 2–6 years of professional experience in legal practice, academia, or policy
- Strong command of legal reasoning, statutory interpretation, and jurisprudential theory
- Bar admission, judicial clerkship, or legal research experience is a strong plus
- Excellent written English and ability to craft precise, well-scoped legal questions
- Expected commitment: 10+ hours/week
- Asynchronous, fully remote work
Requirements
- Must be eligible to work in Remote
- Fluent proficiency in English (Written & Verbal)
- Reliable high-speed internet connection
- Bachelor's's degree or equivalent professional experience
- Demonstrated expertise in Writing
How long hiring takes
Across the AI training platforms we refer candidates to, the median gap between referral and hire is about 30 days. It varies by platform and role, so treat it as a rough guide for this one.
Talent Pool members
Apply through this link and we can put you forward to Mercor when your profile is a strong match. Not every applicant is submitted. If you're not in the pool yet, set up your profile first.
Set up your profile →Why this role
Being remote means this Law Model Prompt Evaluator position pays $75–$100/hr regardless of your zip code, opening up Writing work that used to require living in one of a handful of cities.
Talent pool
We're light on Writing candidates
We've matched 30 people with a Writing background against 397 Writing listings we've tracked, so most go out without one. Set up a profile and we'll consider you for a role like this one.
Set up your profileSkills and categories
Explore other opportunities in related specializations:
Related jobs
Browse All Jobs from Mercor
Discover more opportunities on Mercor that match your skills and interests.
View All Mercor Jobs →Verified Reviews
Community Reviews
Share your experience with Mercor
Help other candidates make better decisions by leaving a review.
Sign in to leave a reviewLeave your review
Common questions
Does Mercor's application require an on-camera interview?
Yes, every applicant records a video interview with an AI interviewer that asks questions about your resume. Clients review that recording to judge communication skills before matching, so there's no way to apply without going on camera.
Does it cost money to apply to Mercor?
No, applying and joining Mercor is free. Mercor's revenue comes from a fee it charges the client on top of your hourly rate, not from applicants. Treat any request for payment to join as a red flag.
What does task-based AI training work look like?
Practical, hands-on data work: recording short videos, categorizing images, rating text responses, or analyzing data. Tasks are designed to be short and distinct, typically 5 to 60 minutes each.
What does Writing work look like for a Law Model Prompt Evaluator?
Tasks here are scoped to Writing, not generic labeling. As a Law Model Prompt Evaluator, expect to draw on real domain judgment (evaluating outputs, correcting errors, or providing expert reasoning specific to Writing) rather than following a one-size-fits-all rubric. If you don't have hands-on Writing background, this is likely not the right listing to start with.
What specific skills does this listing call for?
English and Expert are named directly in the listing. If you don't have hands-on experience with these, expect the screening process to test for them directly rather than accepting adjacent experience as a substitute.
How many hours per week does this role require?
Based on the listing, this role is scoped at about 10 hours per week. Treat this as a real commitment expectation, not a loose estimate.
How much does this specific role pay?
This listing is posted at $75–$100/hr, an hourly rate. The range reflects experience level and negotiated terms, not a placeholder, so where you land in it depends on your background and the assessment. Pay can change between when we last checked the listing and when you apply, so confirm the current number on the platform's own application page before committing time.
What happens when I click Apply on this listing?
You'll be taken to Mercor's external site to complete your application there. This listing links through a referral, but the process is identical to applying directly; the link just routes you correctly. Create an account on their site and follow their onboarding steps.
How soon will I start working after applying to Mercor?
Not immediately. Mercor is a talent marketplace, not a task queue, so applying puts you in a pool of candidates. You start working only once a specific client, like a major AI lab, selects your profile, and that matching process can take weeks.