JSON-fluent & Rubrics Expert
Mercor • Remote
Education
Any
Type
hourly
Pay Rate (by country)
$50–$175/hr
Listed
5d ago
✅ Applying through this link supports our platform at no cost to you.
This position is hosted on an external talent platform. Please only apply for this position if it fits your skills and interests.
In our Talent Pool?
Apply through this link and we can vouch for you to Mercor. ? We vouch for Talent Pool members who apply through our referral link, when we believe they're a strong match. Not every applicant gets a vouch. Not in the pool yet? Set up your profile first.
Set up your profile →Mercor: our referral track record
We've referred 191 candidates to Mercor roles. 14% (27) were placed.
About this Role
From the Mercor listing
About the Opportunity
Mercor is recruiting detail-oriented graders for a weekend sprint beginning today, July 24th. Please apply only if you can begin work immediately! Onboarding happens right away, and the sprint starts on day one.
This project is supporting one of the world’s leading AI research organization - you will assess model outputs against a defined rubric of grading criteria. The ideal candidate has experience processing JSON files, and an understanding of assessing and grading human data tasks against rubrics.
What you’ll do:
You will assess JSON-formatted AI model outputs through the following:
Reading and understanding written task descriptions
Reading and understanding written task rubrics for these tasks
Parsing, reading, and understanding a JSON-based output, and grading its contents against the task rubrics provided
You must complete these assessments without the assistance of AI tools.
Attention to detail and consistency in applying rubric criteria are essential. Prior experience with rubrics is a plus.
Applicants must:
Be available to start today, July 24, running to July 27-28, with possibility for extension later into the week.
Be comfortable reading and interpreting JSON file
Ideal candidates will have prior Mercor experience, especially in applying detailed rubrics and evaluation criteria consistently
Be based in the US
More details about the role:
You must be able to dedicate a minimum of 20-40 hours per week on this task
Turnaround expectation: Ability to complete tasks within 12-24 hours
You will need access to a desktop or laptop computer for this project
We cannot accept applicants who use Chromebook computers
Requirements
- Must be eligible to work in Remote
- Fluent proficiency in English (Written & Verbal)
- Reliable high-speed internet connection
- Bachelor's degree or equivalent professional experience
- Demonstrated expertise in Generalist
Why This Role
Rare opportunity for top 1% experts. Earn $112.5/hr contributing to the world's most advanced AI labs. This is one of the few roles where academic precision is valued as highly as commercial output.
Skills & Categories
Explore other opportunities in related specializations:
Related Jobs
Browse All Jobs from Mercor
Discover more opportunities on Mercor that match your skills and interests.
View All Mercor Jobs →Verified Reviews
Community Reviews
Share your experience with Mercor
Help other candidates make better decisions by leaving a review.
Sign in to leave a reviewLeave your review
Frequently Asked Questions
Is Mercor for freelancers or full-time contractors?
Mercor places you with one client for a defined engagement, like 'Python Tutor for 3 months', rather than having you grab small tasks from a shared queue. Most roles function as steady contract work, not one-off gigs.
Does Mercor's application require an on-camera interview?
Yes, every applicant records a video interview with an AI interviewer that asks questions about your resume. Clients review that recording to judge communication skills before matching, so there's no way to apply without going on camera.
Does it cost money to apply to Mercor?
No, applying and joining Mercor is free. Mercor's revenue comes from a fee it charges the client on top of your hourly rate, not from applicants. Treat any request for payment to join as a red flag.
Is AI training work the same as traditional consulting?
No. Instead of client deliverables, you're given complex scenarios to evaluate: grading the AI's logic, correcting its hallucinations, and supplying expert-level reasoning it doesn't have on its own. The job is closer to teaching than consulting.
Why do these AI training roles pay so much?
Because general knowledge isn't what's being tested. The model already knows the basics; what it needs is expertise on edge cases, the rare, difficult, highly technical judgment calls only a senior professional in the field would make correctly.
What does the day-to-day workload look like for elite-expert AI training roles?
Slow and deep, not fast and repetitive. A single task can take 45-60 minutes of researching citations or verifying complex calculations. Quality is what's being measured here, not throughput.
What does Generalist work look like for a JSON-fluent & Rubrics Expert?
Tasks here are scoped to Generalist, not generic labeling. As a JSON-fluent & Rubrics Expert, expect to draw on real domain judgment (evaluating outputs, correcting errors, or providing expert reasoning specific to Generalist) rather than following a one-size-fits-all rubric. If you don't have hands-on Generalist background, this is likely not the right listing to start with.
What happens when I click Apply on this listing?
You'll be taken to Mercor's external site to complete your application there. This listing links through a referral, but the process is identical to applying directly; the link just routes you correctly. Create an account on their site and follow their onboarding steps.
How soon will I start working after applying to Mercor?
Not immediately. Mercor is a talent marketplace, not a task queue, so applying puts you in a pool of candidates. You start working only once a specific client, like a major AI lab, selects your profile, and that matching process can take weeks.