AI Evaluation Engineer (Python, QA or Security)

General evaluation tasks pay $15-30/hr because they're high-volume, lower-complexity work like basic fact-checking and tone evaluation that doesn't require an advanced degree. Verified domain experts see rates scale up to $40-100+/hr on specialized projects, so the path is to start generalist and build toward a specialist track.

Read a scenario, write a short prompt for the AI (around 100 words), evaluate two AI responses to it, fact-check the outputs, then write a brief explanation (around 50 words) on which response is better against the project rubric. The focus is clarity, safety, and rule-following, not creative writing.

Most skip a live interview entirely and gate access through an automated assessment or qualification task instead. Where we've confirmed the specific requirement for this listing, it's called out above in What We Know About This Role.

Practical, hands-on data work: recording short videos, categorizing images, rating text responses, or analyzing data. Tasks are designed to be short and distinct, typically 5 to 60 minutes each.

Tasks here are scoped to Software Engineering, not generic labeling. As an AI Evaluation Engineer (Python, QA or Security), expect to draw on real domain judgment (evaluating outputs, correcting errors, or providing expert reasoning specific to Software Engineering) rather than following a one-size-fits-all rubric. If you don't have hands-on Software Engineering background, this is likely not the right listing to start with.

AI Training is named directly in the listing. If you don't have hands-on experience with this, expect the screening process to test for it directly rather than accepting adjacent experience as a substitute.

This listing is posted at $40/hr, an hourly rate. Pay can change between when we last checked the listing and when you apply, so confirm the current number on the platform's own application page before committing time.

This specific role is open only to people based in Serbia, Cyprus, Malta, Bosnia and Herzegovina, Lithuania, and Latvia. If you are somewhere else, applying is unlikely to lead to an offer even if you pass the assessment, because the restriction is usually about where the work can legally be contracted rather than your skills. Read the full description for any tax-residency or right-to-work caveats before you apply, since they can differ by country.