Description
Responsibilities
- Create and execute role-play–based evaluation scenarios that simulate realistic customer service interactions across multiple domains, including:
- Flight bookings and travel support
- Financial services
- Telecommunications and technical support
- Contribute to the development of diverse and representative datasets used to assess conversational audio agents.
- Evaluate model performance across a standardized set of qualitative and quantitative metrics.
- Ensure evaluations reflect real customer expectations for clarity, efficiency, and natural conversational flow.
Evaluation Metrics
Model performance is assessed using a combination of conversational, technical, and audio-specific criteria, including but not limited to:
- Task completion accuracy and efficiency
- Conversational naturalness (tone, flow, and coherence)
- Audio comprehension and response quality
- Instruction adherence and contextual understanding
- Basic computer programming literacy, including:
- Understanding of JSON structures
- Familiarity with functions and methods
- Ability to reason about structured data and simple logic
- Technical communication clarity when handling support-style problem-solving
Technical & Equipment Requirements
- Strong verbal communication skills in a simulated customer support context
- English proficiency including fluency across all language skills: reading, listening, writing, and speaking.
- Access to a high-quality microphone to ensure clean, reliable audio input during evaluations
- Comfort working with structured prompts, evaluation rubrics, and technical guidelines
- Device capable of running audio recording software and opening large technical documentation
We offer a pay range of $11-to- $30.65 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. Final offer amounts may vary from the pay range listed above. As a contractor you’ll supply a secure computer and high‑speed internet; company‑sponsored benefits such as health insurance and PTO do not apply.
Employment type: Contract
Workplace type: Remote
Seniority level: Mid‑Senior Level
Similar jobs
Est. 57,000 USD
A project dedicated to assessing and benchmarking advanced agentic audio models against leading systems. The program’s mission is to evaluate and optimize model performance for real-world customer support use cases.Respo…
Est. 52,000 USD
Project Overview We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project assessing advanced agentic audio models. As AI models increasingly handle complex workflows in this domain—…
Project Overview We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project assessing advanced agentic audio models. As AI models increasingly handle complex workflows in this domain—…
Est. 60,000 USD
Project Overview We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project assessing advanced agentic audio models. As AI models increasingly handle complex workflows in this domain—…
Project Overview We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project assessing advanced agentic audio models. As AI models increasingly handle complex workflows in this domain—…
Est. 60,000 USD
A project dedicated to assessing and benchmarking advanced agentic audio models against leading systems. The program’s mission is to evaluate and optimize model performance for real-world customer support use cases. Resp…
Est. 60,000 USD
Project Overview We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project assessing advanced agentic audio models. As AI models increasingly handle complex workflows in this domain—…
Est. 52,000 USD
Project Overview We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project assessing advanced agentic audio models. As AI models increasingly handle complex workflows in this domain—…
Est. 60,000 USD
A project dedicated to assessing and benchmarking advanced agentic audio models against leading systems. The program’s mission is to evaluate and optimize model performance for real-world customer support use cases.Respo…
Est. 80,000 USD
Are you an audio editing expert eager to shape the future of AI? Large‑scale language models are evolving from clever chatbots into powerful engines of multimedia understanding. With high‑quality training data, tomorrow’…
Est. 76,200 USD
Are you an audio editing expert eager to shape the future of AI? Large‑scale language models are evolving from clever chatbots into powerful engines of multimedia understanding. With high‑quality training data, tomorrow’…
Est. 55,000 GBP
Are you an audio editing expert eager to shape the future of AI? Large‑scale language models are evolving from clever chatbots into powerful engines of multimedia understanding. With high‑quality training data, tomorrow’…
Est. 42,000 PLN
Are you an audio editing expert eager to shape the future of AI? Large‑scale language models are evolving from clever chatbots into powerful engines of multimedia understanding. With high‑quality training data, tomorrow’…
Est. 55,000 GBP
Are you a sound engineering expert eager to shape the future of AI? Large‑scale language models are evolving from clever chatbots into powerful tools for audio innovation. With high‑quality training data, tomorrow’s AI c…
Are you a sound engineering expert eager to shape the future of AI? Large‑scale language models are evolving from clever chatbots into powerful tools for audio innovation. With high‑quality training data, tomorrow’s AI c…
Est. 104,000 USD
Are you a sound engineering expert eager to shape the future of AI? Large‑scale language models are evolving from clever chatbots into powerful tools for audio innovation. With high‑quality training data, tomorrow’s AI c…
Est. 55,000 EUR
Are you an experienced voice actor eager to shape the future of AI? Large-scale language models and speech technologies are evolving rapidly, moving beyond simple interactions into expressive, human-like communication. W…
Are you an experienced voice actor eager to shape the future of AI? Large-scale language models and speech technologies are evolving rapidly, moving beyond simple interactions into expressive, human-like communication. W…
Are you an experienced voice actor eager to shape the future of AI? Large-scale language models and speech technologies are evolving rapidly, moving beyond simple interactions into expressive, human-like communication. W…
Est. 52,000 GBP
Are you an experienced voice actor eager to shape the future of AI? Large-scale language models and speech technologies are evolving rapidly, moving beyond simple interactions into expressive, human-like communication. W…
Est. 60,000 USD
Are you an experienced voice actor eager to shape the future of AI? Large-scale language models and speech technologies are evolving rapidly, moving beyond simple interactions into expressive, human-like communication. W…
Est. 62,000 USD
Project Overview We are sourcing independent Vocal Audio Specialists to provide their expertise for an AI audio model training project. As AI models increasingly generate and interpret human musicality and vocal nuances,…
Est. 52,000 USD
Project Overview We are sourcing independent Vocal Audio Specialists to provide their expertise for an AI audio model training project. As AI models increasingly generate and interpret human musicality and vocal nuances,…
Est. 60,000 USD
Are you an experienced Mandarin Chinese voice actor eager to shape the future of AI? Large-scale language models and speech technologies are evolving rapidly, moving beyond simple interactions into expressive, human-like…
Est. 60,000 USD
Are you an experienced Japanese voice actor eager to shape the future of AI? Large-scale language models and speech technologies are evolving rapidly, moving beyond simple interactions into expressive, human-like communi…
Est. 41,600 USD
Are you eager to shape the future of AI? Large-scale language models are transforming the way we learn, work, and communicate—moving beyond simple tasks into tools that can support education, research, and professional p…
Est. 60,000 USD
Are you an experienced Spanish voice actor eager to shape the future of AI? Large-scale language models and speech technologies are evolving rapidly, moving beyond simple interactions into expressive, human-like communic…
Est. 40,000 USD
Are you an experienced voice actor eager to shape the future of AI? Large-scale language models and speech technologies are evolving rapidly, moving beyond simple interactions into expressive, human-like communication. W…
Est. 60,000 USD
Are you an experienced voice actor eager to shape the future of AI? Large-scale language models and speech technologies are evolving rapidly, moving beyond simple interactions into expressive, human-like communication. W…
Est. 120,000 USD
Are you a marketing and sales expert eager to shape the future of AI? Large‑scale language models are evolving from clever chatbots into powerful engines of business insight and customer engagement. With high‑quality tra…