QA Specialist - Audio Annotation & Diarization (US English)

About Turing: Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.

Role Overview:

We are building a highly accurate, evaluation-grade dataset of transcribed, multi-channel audio recordings to assess multilingual, multi-speaker AI systems. This project involves evaluating high-quality and realistic conversations representing diverse dynamics, contexts, and demographics.

The Workflow 

Contributors record unscripted group conversations. These recordings are run through an initial transcription and diarization pass, which contributors then human-validate and correct. As a QA Specialist, you are the final line of defense, responsible for reviewing the end-to-end quality of both the audio recordings and the human-verified annotations.

Key Responsibilities

Audio Quality Assurance:

Evaluate multi-channel audio recordings to ensure they meet strict technical and fidelity requirements. Verify channel isolation (ensuring no audio bleed) and confirm that recordings were captured in appropriate, quiet environments free from disruptive background noise, clipping, or low gain.

Transcription & Diarization Verification:

Review human-validated transcriptions to guarantee exceptionally high accuracy and adherence to strict low error-rate (WER) targets. Confirm that transcripts correctly capture spontaneous, unnormalized speech, preserving natural conversational dynamics such as overlaps, interruptions, and false starts. Validate the precision of turn-level and word-level timestamps, as well as speaker identification, paying special attention to complex, overlapping dialogue , while comfortably reading and validating the underlying JSON-formatted data to ensure accurate metadata tagging and timestamp logic.

Metadata & Content Review:

Verify the accuracy of all applied metadata, including demographic markers, contextual domains, and specific conversational tags. Enforce strict safety and privacy standards by auditing sessions to ensure no Personally Identifying Information (PII), toxic, or sensitive content is present. Execution & Reporting: Assess the end-to-end quality of the annotation task, assigning clear pass/fail or agree/disagree statuses during your review. Provide detailed, actionable comments and feedback whenever disagreeing with an annotator's work.

Requirements & Qualifications

Exceptional ear for audio fidelity and the ability to detect subtle background noises, channel bleed, or clipping. Meticulous attention to detail for verifying word-level timestamps and strict, unnormalized verbatim transcription rules.  Native proficiency in US English Ability to accurately assess complex multi-speaker dynamics.

Ideal Backgrounds include:

Linguists/Phonetics Experts: Deep understanding of natural, unnormalized speech patterns. Expertise in accurately identifying and annotating complex conversational dynamics, including overlaps, false starts, and backchannels. Language Teachers: Exceptional, native-level mastery of the target language. Ability to strictly adhere to verbatim transcription guidelines, documenting every stutter, filler word, and disfluency without applying prescriptive grammar corrections. Professional Transcriptionists: Ear for audio fidelity , rigorous timestamping skills, and experience meeting strict accuracy targets. Rigorous approach to precise turn-level and word-level timestamping.

Perks of Freelancing With Turing:

Work in a fully remote environment. Opportunity to work on cutting-edge AI projects with leading LLM companies. Potential for contract extension based on performance and project needs.

Offer Details:

Commitments Required : at least 4 hours per day and minimum 40 hours per week with 4 hours of overlap with PST. Engagement type  : Contractor assignment/freelancer (no medical/paid leave) Duration of contract : 15 weeks

Evaluation Process :

Shortlisted candidates will be reviewed by our team internally and will be reached out for onboarding.