PhD Rater
Mercor • Remote • Posted 88 days ago
Education
Any
Type
Pay Rate
$95/task
Posted
88d ago
✅ Applying through this link gives you a verified candidate referral.
Referrals from verified candidates give your profile a visibility boost and help support our platform at no cost to you.
This position is hosted on an external talent platform. Please only apply for this position if it fits your skills and interests.
Applying to Mercor?
We support strong candidates applying here. Set up your talent profile so we know who you are.
Set up your profile →About this Role
- Role Overview
Mercor is seeking experienced researchers and technical experts to contribute to a project supporting a frontier-model evaluation effort focused on agentic workflows. You’ll design and validate challenging benchmark tasks in data science, machine learning, finance, and coding to help surface and diagnose reasoning and problem-solving gaps in a target STEM model. The work centers on building robust, real-world tasks with executable tests and then analyzing model/agent behavior.
- Key Responsibilities
Design challenging, real-world STEM problems
Implement each task inside an agentic development environment using Python
- Core Qualifications
Deep expertise in data science, machine learning, finance, and/or Python-based coding
Active or recently graduated PhD (Top U.S.-based school)
Strong research background in frontier STEM topics
Ability to engage reliably for 30+ hours/week, primarily on weekdays
Demonstrated technical output such as high-quality open-source contributions (especially in agentic / LLM tooling ecosystems)
Comfort reading and reasoning about agent behavior traces to diagnose failure modes beyond surface-level errors
- More About the Opportunity
Initial focus area: agentic workflows for STEM tasks
Familiarity with agentic frameworks and OSS ecosystems is helpful (examples include LangChain, MetaGPT, AutoGen, AutoGPT, CrewAI, LlamaIndex, BabyAGI, SuperAGI, CAMEL, AgentGPT, Dify, etc.)
Deliverables are expected to be reproducible and testable (clear specs, deterministic tests where possible, documented environments)
- About Mercor
Mercor is a talent marketplace that connects top experts with leading AI labs and research organizations.
Our investors include Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey.
Thousands of professionals across domains like law, creatives, engineering, and research have joined Mercor to work on frontier projects shaping the next era of AI.
We consider all qualif
Requirements
- Must be eligible to work in Remote
- Fluent proficiency in English (Written & Verbal)
- Reliable high-speed internet connection
- Bachelor's degree or equivalent professional experience
- Demonstrated expertise in Data Science
Compensation Analysis
Monetize your niche expertise without the billable hours. At $95/hr, this role offers elite compensation for pure intellectual work—no client management or administrative bloat.
Skills & Categories
Explore other opportunities in related specializations:
Related Jobs
Browse All Jobs from Mercor
Discover more opportunities on Mercor that match your skills and interests.
View All Mercor Jobs →Community Reviews
Share your experience with Mercor
Help other candidates make better decisions by leaving a review.
Sign in to leave a reviewLeave your review
Frequently Asked Questions
Is this for freelancers or full-time employees?
Both. Mercor tries to match you with clients who want long-term contractors. Unlike other platforms where you log in and grab small tasks, Mercor matches you with one company for a steady role (e.g., 'Python Tutor for 3 months').
I'm not comfortable on camera. Can I still apply?
No. The application requires a video interview with an AI avatar. The AI asks you questions about your resume, and the video is shared with potential clients to prove your communication skills.
Does it cost money to join?
No. You should never pay to join these platforms. Mercor makes money by charging the client a fee on top of your hourly rate.
Is this traditional consulting?
Not exactly. You act as a "Teacher" for advanced AI. Instead of client deliverables, you are given complex scenarios to evaluate. You grade the AI's logic, correct its hallucinations, and provide expert-level reasoning. Your job is to train the model to think like you do.
Why is the pay so high?
This role requires deep, verified expertise. General knowledge isn't enough; the model is specifically being trained on "edge cases"—the rare, difficult, or highly technical nuances that only a senior professional would know.
What is the workload like?
This is cognitive, deep work. Unlike simple data labeling, you might spend 45-60 minutes on a single task, researching citations or verifying complex calculations. Quality is prioritized over speed.
How soon will I start?
Important: Mercor is a talent marketplace, not a task queue. Applying puts you in a pool of candidates. You will only start working when a specific client (like a major AI lab) selects your profile. This matching process can take weeks.