Senior Software Engineer — AI Code Ranking
Alignerr • Remote • Posted 4 days ago
Education
Any
Type
Pay Rate
$100.5/task
Posted
4d ago
✅ Applying through this link gives you a verified candidate referral.
Referrals from verified candidates give your profile a visibility boost and help support our platform at no cost to you.
This position is hosted on an external talent platform. Please only apply for this position if it fits your skills and interests.
About this Role
What You'll Do
- Compare pairs of AI-generated code solutions and rank them based on quality, correctness, and engineering best practices
- Evaluate code for functional accuracy — verifying that solutions produce correct outputs across standard and edge cases
- Assess code quality factors including readability, efficiency, maintainability, and adherence to idiomatic patterns
- Identify bugs, logic errors, security issues, and performance bottlenecks in AI-generated code
- Write clear, structured justifications explaining your ranking decisions
- Work across multiple programming languages and problem domains — from algorithms and data structures to systems design and API integrations
- Complete task-based assignments independently on your own schedule
About the Role
What if your deep software engineering expertise could directly shape how the next generation of AI writes code? We're looking for senior-level developers to evaluate, compare, and rank AI-generated code — helping AI systems learn what separates good code from great code. You'll review pairs of code solutions, assess them on correctness, efficiency, readability, and best practices, and provide the preference rankings that teach AI models to produce better software. Your engineering judgment becomes the training signal that makes AI smarter. This is a fully remote, flexible contract role designed for experienced software engineers who want meaningful, intellectually engaging work on their own schedule.
- Organization: Alignerr
- Type: Hourly Contract
- Location: Remote
- Commitment: 10–40 hours/week
Who You Are
- Senior-level software engineer with 5+ years of professional development experience
- Proficient in at least two of the following: Python, JavaScript/TypeScript, Java, C++, Go, or Rust
- Strong understanding of algorithms, data structures, and computational complexity
- Experienced in code review — you can quickly spot issues and articulate what makes one solution better than another
- Deep knowledge of software engineering best practices: clean code, design patterns, testing, and performance optimization
- Clear written communicator who can explain technical reasoning concisely
- Self-motivated and consistent when working independently without supervision
Nice to Have
- Experience with machine learning, AI/ML pipelines, or training data curation
- Background in competitive programming, technical interviewing, or algorithmic problem-solving
- Familiarity with RLHF (Reinforcement Learning from Human Feedback) or preference-based AI training
- Experience across both backend and frontend development
- Contributions to open-source projects or technical mentoring experience
- Knowledge of security best practices and common vulnerability patterns
Why Join Us
- Work on cutting-edge AI projects alongside leading research labs
- Fully remote and flexible — work when and where it suits you
- Freelance autonomy with the structure of meaningful, task-based work
- Your engineering expertise directly shapes how AI understands and generates code at scale
- Intellectually stimulating work that keeps your skills sharp across languages and domains
- Potential for ongoing work and contract extension as new projects launch
Requirements
- Fluent proficiency in English (Written & Verbal)
- Reliable high-speed internet connection
- Bachelor's degree or equivalent professional experience
- Demonstrated expertise in Software Engineering
Compensation Analysis
What if your deep software engineering expertise could directly shape how the next generation of AI writes code? We're looking for senior-level developers to evaluate, compare, and rank AI-generated code — helping AI systems learn what separates good code from great code. You'll review pairs of code solutions, assess them on correctness, efficiency
Skills & Categories
Explore other opportunities in related specializations:
Related Jobs
Browse All Jobs from Alignerr
Discover more opportunities on Alignerr that match your skills and interests.
View All Alignerr Jobs →Community Reviews
Leave your review
Frequently Asked Questions
What is the assessment actually like?
Notoriously strict. Alignerr uses TestGorilla for role-specific timed tests — a blank coding environment for engineers, rigorous grammar and fact-checking for writers. There is almost no hand-holding. The critical catch: this is essentially a one-shot process. Fail or abandon the assessment, and you are typically locked out of that role permanently with no option to retake.
How quickly can I start earning after I pass?
Not immediately. Even after passing the assessment and completing identity verification (via Persona) and billing setup (via Deel), you may sit in a waiting pool for weeks or months. You only start earning when a project matching your specific skills launches and you are officially assigned. Do not plan around Alignerr income until you are actively on a project.
Is there a community?
Yes — and it is one of Alignerr's genuine strengths. Once assigned to a project, you are added to Slack channels where you can ask questions, get rubric clarifications from admins, and talk to other AI trainers. This is rare in AI training and makes a real difference when guidelines are ambiguous or change mid-project.
Is this traditional consulting?
Not exactly. You act as a "Teacher" for advanced AI. Instead of client deliverables, you are given complex scenarios to evaluate. You grade the AI's logic, correct its hallucinations, and provide expert-level reasoning. Your job is to train the model to think like you do.
Why is the pay so high?
This role requires deep, verified expertise. General knowledge isn't enough; the model is specifically being trained on "edge cases"—the rare, difficult, or highly technical nuances that only a senior professional would know.
What is the workload like?
This is cognitive, deep work. Unlike simple data labeling, you might spend 45-60 minutes on a single task, researching citations or verifying complex calculations. Quality is prioritized over speed.
What is the barrier to entry?
Alignerr is known for difficult technical assessments. You must pass a timed test in your specific domain (e.g., Python, Physics, or Language) before you are eligible for any paid projects.