Law Model Prompt Evaluator
Mercor • Remote
Education
Any
Type
hourly
Pay Rate (by country)
$75–$100/hr
Listed
110d ago
✅ Applying through this link supports our platform at no cost to you.
This position is hosted on an external talent platform. Please only apply for this position if it fits your skills and interests.
In our Talent Pool?
Apply through this link and we can vouch for you to Mercor. ? We vouch for Talent Pool members who apply through our referral link, when we believe they're a strong match. Not every applicant gets a vouch. Not in the pool yet? Set up your profile first.
Set up your profile →Mercor: our referral track record
We've referred 191 candidates to Mercor roles. 14% (27) were placed.
What We Know About This Role
- Weekly hours
- 10 hrs/week
About this Role
From the Mercor listing
Role Overview
We are seeking expert legal professionals to author and verify high-quality open-ended prompts for AI model evaluation. You will craft and review challenging, unambiguous legal problems across core subdomains, assessing AI reasoning quality and helping establish rigorous evaluation standards for frontier language models.
- Authoring Task — Create 5 original, open-ended prompts from your assigned subdomain at varying difficulty levels (undergraduate, advanced undergraduate, or graduate/professional). Prompts should require human judgment to evaluate the quality of the AI's response, such as legal reasoning, statutory interpretation, or case analysis.
- Verification Task — Review 5 authored prompts for clarity, scope alignment, difficulty accuracy, and uniqueness. Edit prompts and difficulty ratings where needed.
- Author clear, unambiguous, open-ended legal prompts that elicit evaluable AI responses
- Verify prompts are within the scope of the assigned subdomain and correctly rated for difficulty
- Ensure all 5 prompts in a task are sufficiently distinct from one another with varying difficulty levels
- Apply expert judgment to assess the depth and quality of legal reasoning required
- Edit prompts and difficulty assignments where standards are not met
- JD, LLM, SJD, or Master's degree or higher in Law or a closely related field
- 2–6 years of professional experience in legal practice, academia, or policy
- Strong command of legal reasoning, statutory interpretation, and jurisprudential theory
- Bar admission, judicial clerkship, or legal research experience is a strong plus
- Excellent written English and ability to craft precise, well-scoped legal questions
- Expected commitment: 10+ hours/week
- Asynchronous, fully remote work
Requirements
- Must be eligible to work in Remote
- Fluent proficiency in English (Written & Verbal)
- Reliable high-speed internet connection
- Bachelor's degree or equivalent professional experience
- Demonstrated expertise in Writing
Eligible Languages
Fluent proficiency in English
Why This Role
Work from anywhere, at any time. This fully remote Law Model Prompt Evaluator position ($87.5/hr) breaks down geographic barriers, allowing you to earn US-competitive rates regardless of your local market. It is a perfect stepping stone for building a career in the Writing ecosystem.
Skills & Categories
Explore other opportunities in related specializations:
Related Jobs
Browse All Jobs from Mercor
Discover more opportunities on Mercor that match your skills and interests.
View All Mercor Jobs →Verified Reviews
Community Reviews
Share your experience with Mercor
Help other candidates make better decisions by leaving a review.
Sign in to leave a reviewLeave your review
Frequently Asked Questions
Is Mercor for freelancers or full-time contractors?
Mercor places you with one client for a defined engagement, like 'Python Tutor for 3 months', rather than having you grab small tasks from a shared queue. Most roles function as steady contract work, not one-off gigs.
Does Mercor's application require an on-camera interview?
Yes, every applicant records a video interview with an AI interviewer that asks questions about your resume. Clients review that recording to judge communication skills before matching, so there's no way to apply without going on camera.
Does it cost money to apply to Mercor?
No, applying and joining Mercor is free. Mercor's revenue comes from a fee it charges the client on top of your hourly rate, not from applicants. Treat any request for payment to join as a red flag.
What does task-based AI training work actually look like?
Practical, hands-on data work: recording short videos, categorizing images, rating text responses, or analyzing data. Tasks are designed to be short and distinct, typically 5 to 60 minutes each.
What does asynchronous AI training work mean in practice?
No set hours, no check-ins, no meetings. You log in when you want, pick up an available task, complete it, and submit; nobody is waiting on you in real time. That's different from remote employment, where you're expected online during business hours. The tradeoff: you're competing with others for available tasks, so an empty queue means there's simply nothing to do until more work is released.
What does Writing work look like for a Law Model Prompt Evaluator?
Tasks here are scoped to Writing, not generic labeling. As a Law Model Prompt Evaluator, expect to draw on real domain judgment (evaluating outputs, correcting errors, or providing expert reasoning specific to Writing) rather than following a one-size-fits-all rubric. If you don't have hands-on Writing background, this is likely not the right listing to start with.
How many hours per week does this role require?
Based on the listing, this role is scoped at about 10 hours per week. Treat this as a real commitment expectation, not a loose estimate.
Do I need to be fluent in English?
Yes. This role specifically requires English proficiency. You will likely be evaluated on written fluency during the assessment, not just conversational level. If English is not your first language or you are not professionally fluent, this is not the right role. Filter for your native language to find better-matched listings.
What happens when I click Apply on this listing?
You'll be taken to Mercor's external site to complete your application there. This listing links through a referral, but the process is identical to applying directly; the link just routes you correctly. Create an account on their site and follow their onboarding steps.
How soon will I start working after applying to Mercor?
Not immediately. Mercor is a talent marketplace, not a task queue, so applying puts you in a pool of candidates. You start working only once a specific client, like a major AI lab, selects your profile, and that matching process can take weeks.