AI Agent Evaluation Analyst (Freelance)
Mindrift • United States
Education
Not stated
Type
Hourly
Pay Rate
$80/hr
Listed
271d ago
About this role
From the Mindrift listing
This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency. At Mindrift , innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI. What we do The Mindrift platform, launched and powered by Toloka , connects domain experts with cutting-edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real-world expertise from across the globe. Who we're looking for: We’re looking for curious and intellectually proactive contributors, the kind of person who double-checks assumptions and plays devil’s advocate. Are you comfortable with ambiguity and complexity? Does an async, remote, flexible opportunity sound exciting? Would you like to learn how modern AI systems are tested and evaluated? This is a flexible, project-based opportunity well-suited for: Analysts, researchers, or consultants with strong critical thinking skills. Students (senior undergrads / grad students) looking for an intellectually interesting gig. People open to a part-time and non-permanent opportunity. About the project: We’re on the hunt for QAs for autonomous AI agents for a new project focused on validating and improving complex task structures, policy logic, and agent evaluation frameworks. Throughout the project, you’ll have to balance quality assurance, research, and logical problem-solving. This project opportunity is ideal for people who enjoy looking at systems holistically and thinking through scenarios, implications, and edge cases. You do not need a coding background, but you must be curious, intellectually rigorous, and capable of evaluating the soundness and consistency of complex setups. If you’ve ever excelled in things like consulting, CHGK, Olympiads, case solving, or systems thinking — you might be a great fit. What you’ll be doing: Reviewing evaluation tasks and scenarios for logic, completeness, and realism. Identifying inconsistencies, missing assumptions, or unclear decision points. Helping define clear expected behaviors (gold standards) for AI agents. Annotating cause-effect relationships, reasoning paths, and plausible alternatives. Thinking through complex systems and policies as a human would to ensure agents are tested properly. Working closely with QA, writers, or developers to suggest refinements or edge case coverage. How to get started: Apply to this post, qualify, and get the chance to contribute to a project aligned with your skills, on your own schedule. Shape the future of AI while building tools that benefit everyone.
Requirements
- Must be eligible to work in United States
- Fluent proficiency in English (Written & Verbal)
- Reliable high-speed internet connection
- Bachelor's degree or equivalent professional experience
- Demonstrated expertise in Generalist
Why this role
This AI Agent Evaluation Analyst (Freelance) role converts Generalist background into $80/hr of AI training work, with onboarding covering whatever AI-specific process is new to you.
Skills and categories
Explore other opportunities in related specializations:
Related jobs
Browse All Jobs from Mindrift
Discover more opportunities on Mindrift that match your skills and interests.
View All Mindrift Jobs →Verified Reviews
Community Reviews
Share your experience with Mindrift
Help other candidates make better decisions by leaving a review.
Sign in to leave a reviewLeave your review
Common questions
What does a typical Mindrift task involve?
Read a scenario, write a short prompt for the AI (around 100 words), evaluate two AI responses to it, fact-check the outputs, then write a brief explanation (around 50 words) on which response is better against the project rubric. The focus is clarity, safety, and rule-following, not creative writing.
Who is Mindrift good for?
Freelance writers, editors, and generalist AI tutors with strong English fluency and solid research skills, no specialized tech degree required. Built by data-labeling company Toloka, Mindrift also opens higher-paying specialized projects (cybersecurity, medicine, law) to domain experts once verified.
What does task-based AI training work look like?
Practical, hands-on data work: recording short videos, categorizing images, rating text responses, or analyzing data. Tasks are designed to be short and distinct, typically 5 to 60 minutes each.
What does asynchronous AI training work mean in practice?
No set hours, no check-ins, no meetings. You log in when you want, pick up an available task, complete it, and submit; nobody is waiting on you in real time. That's different from remote employment, where you're expected online during business hours. The tradeoff: you're competing with others for available tasks, so an empty queue means there's simply nothing to do until more work is released.
What does Generalist work look like for an AI Agent Evaluation Analyst (Freelance)?
Tasks here are scoped to Generalist, not generic labeling. As an AI Agent Evaluation Analyst (Freelance), expect to draw on real domain judgment (evaluating outputs, correcting errors, or providing expert reasoning specific to Generalist) rather than following a one-size-fits-all rubric. If you don't have hands-on Generalist background, this is likely not the right listing to start with.
What specific skills does this listing call for?
AI Training and Review & QA are named directly in the listing. If you don't have hands-on experience with these, expect the screening process to test for them directly rather than accepting adjacent experience as a substitute.
How much does this specific role pay?
This listing is posted at $80/hr, an hourly rate. Pay can change between when we last checked the listing and when you apply, so confirm the current number on the platform's own application page before committing time.
Can I apply from outside the United States?
This specific role is open only to people based in the United States. If you are somewhere else, applying is unlikely to lead to an offer even if you pass the assessment, because the restriction is usually about where the work can legally be contracted rather than your skills. Read the full description for any tax-residency or right-to-work caveats before you apply.