AI Evaluation Engineer (Python / Java / Web)
Turing • Remote
Education
Any
Type
hourly
Pay Rate (by country)
$25–$55/hr
Listed
55d ago
✅ Applying through this link supports our platform at no cost to you.
This position is hosted on an external talent platform. Please only apply for this position if it fits your skills and interests.
Turing: our referral track record
We've referred 677 candidates to Turing roles. 2% (15) were placed.
Turing: typical time to hire
Based on 15 tracked placements across all Turing roles on our site.
What We Know About This Role
- Interview
- Required i As stated in the listing: “Duration of contract : 2 months”
- Weekly hours
- 40 hrs/week
- Timezone overlap
- Partial overlap with US Pacific time required
- Contract length
- 8 weeks
About this Role
From the Turing listing
About Turing
Turing is one of the world's fastest-growing AI companies accelerating the advancement and deployment of powerful AI systems.
Turing helps customers in two ways: working with the world's leading AI labs to advance frontier model capabilities and leveraging that work to build real-world AI systems that help businesses solve complex problems and unlock new opportunities.
Role Overview
We are seeking detail-oriented software professionals with expertise in Java and Web Development to join Turing's AI Evaluation team as AI Evaluation Specialists.
In this role, you will design and create evaluation tasks that measure the capabilities of advanced AI systems on real-world software engineering challenges. You will author structured tasks, reference solutions, verification criteria, and domain-specific guidance used to evaluate how effectively AI models perform development-related work.
Your contributions will directly influence the benchmarking and improvement of next-generation AI systems used by leading AI organizations.
What does day-to-day look like? Design realistic software engineering evaluation tasks focused on Java and Web Development Write clear, precise instructions that define expected outputs, constraints, and success criteria for AI agents Create reference solutions that successfully solve evaluation tasks and satisfy validation requirements Develop human-readable verifier descriptions that document expected behaviors and evaluation checks Author domain-specific knowledge files that teach AI systems relevant workflows, conventions, and best practices without revealing solutions Review tasks for clarity, consistency, edge cases, and evaluation quality Analyze AI-generated outputs and identify common failure patterns Collaborate with researchers and evaluation teams to improve benchmark quality and coverage Follow established task-authoring standards and quality guidelines Requirements Bachelor's degree or higher in Computer Science, Software Engineering, Information Technology, or a related field 3–5 years of hands-on software development experience Strong proficiency in Java and core software engineering concepts Experience building or maintaining web applications (Frontend, Backend, or Full Stack) Strong understanding of object-oriented programming, APIs, web technologies, and software development best practices Excellent written English with the ability to write clear, precise, and unambiguous technical instructions Ability to think critically about how AI systems interpret instructions and solve technical problems Comfortable working with structured formats such as JSON, Markdown, YAML, DOCX, and XLSX
Nice to have:
Experience with LLM evaluation, prompt engineering, or AI benchmarking Experience creating technical assessments, coding challenges, or educational content Experience with Docker, containers, or cloud-based development environments Perks of Freelancing With Turing Work on cutting-edge AI projects with leading AI research organizations Flexible remote work opportunities Opportunity to influence the evaluation of next-generation AI systems Collaborate with a global network of highly skilled professionals Offer details: Commitments Required : 40 hours per week with overlap of 4 hours with PST Engagement type : Contractor assignment/freelancer (no medical/paid leave) Duration of contract : 2 months; [expected start date is next week] Evaluation Process One round of technical interview (or) Automated Live coding challenge
Requirements
- Java
- JavaScript
- Must be eligible to work in Remote
What to Expect
Looking at Turing STEM listings we've tracked, contracts in this domain typically run about 9.2 weeks. Actual length varies by project, but this gives you a realistic baseline going in. This listing's 8-week contract is shorter than the typical 9.2-week Turing STEM engagement.
Based on 19 extracted Turing STEM listings.
Eligible Languages
Fluent proficiency in English
Why This Role
Geography stops mattering here. This remote AI Evaluation Engineer (Python / Java / Web) opportunity pays $40/hr regardless of where you're based, making it a solid entry point into STEM work without needing to relocate.
Skills & Categories
Explore other opportunities in related specializations:
Related Jobs
Browse All Jobs from Turing
Discover more opportunities on Turing that match your skills and interests.
View All Turing Jobs →Verified Reviews
Community Reviews
Share your experience with Turing
Help other candidates make better decisions by leaving a review.
Sign in to leave a reviewLeave your review
Frequently Asked Questions
Do I need to be a software engineer to work for Turing?
No, not anymore. Turing built its name matching senior engineers with Silicon Valley companies, but it has since expanded into AGI infrastructure work and now hires non-engineering domain experts, technical writers, and researchers for post-training data annotation and RLHF. A strong analytical background and excellent English matter more than coding ability.
How does Turing's talent matching work?
Turing calls it the Intelligent Talent Cloud. You build a profile and go through vetting (automated tests, an AI-powered interview, practical skill assessments), and once vetted, Turing's algorithm surfaces your profile directly to partner companies like Fortune 500s and top AI labs. You don't browse listings or bid on work; matches come to you.
How and when does Turing pay contractors?
Monthly, in USD, via Deel, Payoneer, or direct bank transfer. You're engaged as an independent contractor responsible for your own local taxes. Plan your cash flow around a monthly cycle if you're used to weekly payouts elsewhere.
What does task-based AI training work actually look like?
Practical, hands-on data work: recording short videos, categorizing images, rating text responses, or analyzing data. Tasks are designed to be short and distinct, typically 5 to 60 minutes each.
What does asynchronous AI training work mean in practice?
No set hours, no check-ins, no meetings. You log in when you want, pick up an available task, complete it, and submit; nobody is waiting on you in real time. That's different from remote employment, where you're expected online during business hours. The tradeoff: you're competing with others for available tasks, so an empty queue means there's simply nothing to do until more work is released.
What does STEM work look like for a AI Evaluation Engineer (Python / Java / Web)?
Tasks here are scoped to STEM, not generic labeling. As a AI Evaluation Engineer (Python / Java / Web), expect to draw on real domain judgment (evaluating outputs, correcting errors, or providing expert reasoning specific to STEM) rather than following a one-size-fits-all rubric. If you don't have hands-on STEM background, this is likely not the right listing to start with.
How many hours per week does this role require?
Based on the listing, this role is scoped at about 40 hours per week, over roughly a 8-week contract. Treat this as a real commitment expectation, not a loose estimate.
Do I need to be fluent in English?
Yes. This role specifically requires English proficiency. You will likely be evaluated on written fluency during the assessment, not just conversational level. If English is not your first language or you are not professionally fluent, this is not the right role. Filter for your native language to find better-matched listings.
What happens when I click Apply on this listing?
You'll be taken to Turing's external site to complete your application there. This listing links through a referral, but the process is identical to applying directly; the link just routes you correctly. Create an account on their site and follow their onboarding steps.
Is there an interview?
Yes. One round of technical interview (or) Automated Live coding challenge