Code-Data Eval Author — Software Test Engineer / SDET (Pilot)
Mercor • Remote • Posted 0 days ago
Education
Any
Type
Pay Rate
$65/task
Posted
0d ago
✅ Applying through this link gives you a verified candidate referral.
Referrals from verified candidates give your profile a visibility boost and help support our platform at no cost to you.
This position is hosted on an external talent platform. Please only apply for this position if it fits your skills and interests.
Applying to Mercor?
We support strong candidates applying here. Set up your talent profile so we know who you are.
Set up your profile →About this Role
Code-Data Eval Author — Software Test Engineer / SDET (Mercor · remote contract)
Mercor partners with frontier AI labs to build the evaluations their coding models are trained and measured against. You'll design the verifiers, correctness rubrics, and adversarial test cases that decide whether an AI agent's code actually works.
Location Requirements
Code-Data Eval Author — Software Test Engineer / SDET
What you'll do
You are
Engagement & pay
Hiring process — paid
Code-Data Eval Author — Software Test Engineer / SDET (Mercor · remote contract) Mercor partners with frontier AI labs to build the evaluations their coding models are trained and measured against. You'll design the verifiers, correctness rubrics, and adversarial test cases that decide whether an AI agent's code actually works. What you'll do Design verifiers and correctness rubrics for coding tasks Enumerate edge cases and build adversarial test cases for agent/model evaluation Grade agent trajectories and improve test/rubric quality through review ~5+ years as an SDET / software test engineer at a real product organization Write code and tests: automation frameworks (pytest, Playwright, Cypress), CI/CD (SDET preferred over manual-only QA) Clear written communication; familiarity with AI tools / evals is a plus Engagement & pay Remote contract, flexible 30+ hrs/week Hourly rate set to your local market (e.g., US/Canada $75–100/hr; Europe and LatAm scaled to region) Hiring process — paid A short Mercor Technical Screen, a live Code Review Session, and a Domain Expert Interview. You're paid $200 for completing all three, regardless of outcome. We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
- Design verifiers and correctness rubrics for coding tasks
- Enumerate edge cases and build adversarial test cases for agent/model evaluation
- Grade agent trajectories and improve test/rubric quality through review
- ~5+ years as an SDET / software test engineer at a real product organization
- Write code and tests: automation frameworks (pytest, Playwright, Cypress), CI/CD (SDET preferred over manual-only QA)
- Clear written communication; familiarity with AI tools / evals is a plus
- Remote contract, flexible 30+ hrs/week
- Hourly rate set to your local market (e.g., US/Canada $75–100/hr; Europe and LatAm scaled to region)
Requirements
- Must be eligible to work in Remote
- Fluent proficiency in English (Written & Verbal)
- Reliable high-speed internet connection
- Bachelor's degree or equivalent professional experience
- Demonstrated expertise in STEM
Compensation Analysis
Work from anywhere, at any time. This fully remote position ($65/hr) breaks down geographic barriers, allowing you to earn US-competitive rates regardless of your local market. It is a perfect stepping stone for building a career in the data labeling and AI training ecosystem.
Skills & Categories
Explore other opportunities in related specializations:
Related Jobs
Browse All Jobs from Mercor
Discover more opportunities on Mercor that match your skills and interests.
View All Mercor Jobs →Community Reviews
Share your experience with Mercor
Help other candidates make better decisions by leaving a review.
Sign in to leave a reviewLeave your review
Frequently Asked Questions
Is this for freelancers or full-time employees?
Both. Mercor tries to match you with clients who want long-term contractors. Unlike other platforms where you log in and grab small tasks, Mercor matches you with one company for a steady role (e.g., 'Python Tutor for 3 months').
I'm not comfortable on camera. Can I still apply?
No. The application requires a video interview with an AI avatar. The AI asks you questions about your resume, and the video is shared with potential clients to prove your communication skills.
Does it cost money to join?
No. You should never pay to join these platforms. Mercor makes money by charging the client a fee on top of your hourly rate.
Is this just labeling data?
No. This is closer to academic research. You will likely be writing or verifying complex proofs, solving advanced equations, or checking the logic of a model's step-by-step reasoning. The goal is to teach AI systems to reason deeply in your field.
Do I need a PhD?
For the highest pay tiers in this category, a PhD (or current enrollment) is usually expected. However, the most important factor is your ability to pass the domain assessment. If you can solve the problems, the degree is secondary.
Is the work continuous?
Work in niche fields is often project-based. A specific "campaign" (e.g., training a model on Quantum Mechanics) might last for a few weeks. It is best to treat this as a high-paying fellowship or grant rather than a permanent daily job.
How soon will I start?
Important: Mercor is a talent marketplace, not a task queue. Applying puts you in a pool of candidates. You will only start working when a specific client (like a major AI lab) selects your profile. This matching process can take weeks.