Italian Audio Evaluations Specialist - Freelance AI Trainer Project

About this role

<h3><strong>Project Overview</strong></h3> <p>We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project assessing advanced agentic audio models. As AI models increasingly handle complex workflows in this domain—specifically real-world customer support scenarios like flight bookings, financial services, and telecommunications—their accuracy relies entirely on robust, expert-crafted training data. The objective of this project is to autonomously produce high-quality evaluation tasks through simulated interactions, audit conversational AI outputs, and generate clean, reliable datasets to optimize model performance.</p> <h3><strong>Project Deliverables & Scope</strong></h3> <p>Operate autonomously to design complex evaluation frameworks and provide structured training data. Expected deliverables include:</p> <ul> <li><strong>Role-Play Scenario Execution:</strong> Creating and executing complex, role-play-based evaluation scenarios that simulate realistic customer service interactions across travel, finance, and technical support domains.</li> <li><strong>Model Performance Auditing:</strong> Evaluating AI model performance across standardized qualitative and quantitative metrics, focusing strictly on task completion accuracy, conversational naturalness, and audio comprehension.</li> <li><strong>Technical Metric Evaluation:</strong> Assessing the model's basic computer programming literacy, including its understanding of JSON structures, functions, methods, and ability to reason about structured data within a support context.</li> <li><strong>Representative Dataset Generation:</strong> Contributing to the development of diverse, high-quality audio datasets that accurately reflect real customer expectations for clarity, efficiency, and natural conversational