MCP & Tools Python Developer - Agent Evaluation Infrastructure

Read a scenario, write a short prompt for the AI (around 100 words), evaluate two AI responses to it, fact-check the outputs, then write a brief explanation (around 50 words) on which response is better against the project rubric. The focus is clarity, safety, and rule-following, not creative writing.

Freelance writers, editors, and generalist AI tutors with strong English fluency and solid research skills, no specialized tech degree required. Built by data-labeling company Toloka, Mindrift also opens higher-paying specialized projects (cybersecurity, medicine, law) to domain experts once verified.

Practical, hands-on data work: recording short videos, categorizing images, rating text responses, or analyzing data. Tasks are designed to be short and distinct, typically 5 to 60 minutes each.

No set hours, no check-ins, no meetings. You log in when you want, pick up an available task, complete it, and submit; nobody is waiting on you in real time. That's different from remote employment, where you're expected online during business hours. The tradeoff: you're competing with others for available tasks, so an empty queue means there's simply nothing to do until more work is released.

Tasks here are scoped to Software Engineering, not generic labeling. As a MCP & Tools Python Developer - Agent Evaluation Infrastructure, expect to draw on real domain judgment (evaluating outputs, correcting errors, or providing expert reasoning specific to Software Engineering) rather than following a one-size-fits-all rubric. If you don't have hands-on Software Engineering background, this is likely not the right listing to start with.

AI Training is named directly in the listing. If you don't have hands-on experience with this, expect the screening process to test for it directly rather than accepting adjacent experience as a substitute.

This listing is posted at $80/hr, an hourly rate. Pay can change between when we last checked the listing and when you apply, so confirm the current number on the platform's own application page before committing time.

This specific role is open only to people based in the United States. If you are somewhere else, applying is unlikely to lead to an offer even if you pass the assessment, because the restriction is usually about where the work can legally be contracted rather than your skills. Read the full description for any tax-residency or right-to-work caveats before you apply.