[All remote jobs](https://jobicy.com/jobs.md)[![Cantina Labs logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2026/11/e52bb5e84323-221.webp)](https://jobicy.com/company/cantina-labs.md)[Cantina Labs](https://jobicy.com/company/cantina-labs.md) # Machine Learning Engineer, Core Evaluations AI Summary Cantina Labs seeks a senior Machine Learning Engineer to establish and lead evaluation for speech generation and recognition models. The role designs production and development evaluation pipelines, measurable metrics, subjective user studies, and automated reporting dashboards. The engineer will train or fine-tune evaluation models, apply statistical analysis, and partner with model, data, and product teams to improve model quality. As the founding evaluation-team member, this person will shape technical standards and is expected to grow into a leadership role. The work centers on large-scale ASR and TTS systems, including models exceeding 3B parameters and datasets with more than one million hours of audio.[Apply for this job](#job-application)[View company](https://jobicy.com/company/cantina-labs.md)ShareRemote from[USA](https://jobicy.com/job-region/usa.md), [Europe](https://jobicy.com/job-region/europe.md)SalaryUndisclosedDepartment[Data Science & Analytics](https://jobicy.com/categories/data-science.md)EmploymentFull TimeExperienceSeniorPublished10 Nov 2026Apply before10 Dec 2026Listing views59Application actions4Application toolkit ## Make your next move. Prepare your resume, explore your fit, and draft a cover letter for this opportunity. Opportunity details ## About this role. ### About Cantina: [Cantina Labs](https://jobicy.com/company/cantina-labs.md) is a social AI company, developing a suite of advanced real-time models that push the boundaries of expression, personality, and realism. We bring characters to life, transforming how people tell stories, connect, and create. We build and power ecosystems. Cantina, our flagship social AI platform, is just the beginning. If you’re excited about the potential AI has to shape human creativity and social interactions, join us in building the future! ### About the Role: We are seeking an experienced Machine Learning Engineer (MLE) to focus on audio model evaluation, specifically for speech generation and recognition models. This role involves designing and developing comprehensive model evaluation pipelines for both development and production environments, as well as creating automated dashboards for reporting evaluation results. As the founding member of our evaluation team, the ideal candidate is expected to leverage their experience to lead our evaluation efforts and play a key role in the future growth of the evaluation team. ### What You’ll Do: * Designing model evaluation pipelines for models in development and production * Designing user studies for subjective model evaluations. * Converting requirements into measurable metrics. * Designing and developing automated evaluation dashboard to see model performances and compare results. * Training new models to capture new and different evaluation metrics. * Communicating with the model team to help design better models based on the evaluation results. * Communicating with the data team to help decide the type of data necessary to improve model performance. * Communication with the product-manager to make sure product requirements are correctly measured. * Help grow the evaluation team as the founding member. * Lead the evaluation team in the future. ### What You’ll Bring: * Strong experience and intuition for designing metrics that capture model performance. * Strong experience with designing user studies on Mechanical Turk or similar platforms. . * Strong experience with model training and fine-tuning for model evaluation. * Strong statistical knowledge and experience to statistically compare evaluation results and take decisions. * Very strong engineering and programming skills. * Experience with training ASR, TTS models. * Experience at ML teams working on large-scale machine learning problems. (>3B models with >1m hours of data) ### Compensation: When determining compensation, a number of factors will be considered, including skills, experience, job scope, location, and competitive compensation market data. ### Benefits for U.S.-based roles: * Competitive salary and generous company equity * Medical, dental, and vision insurance – 99.99% of premiums covered by Cantina * 42 days of paid time off, including: 15 PTO days * 10 sick days * 15 company holidays * 2 floating holidays * Generous parental leave & fertility support * 401(k) retirement savings plan * Lifestyle spending account – $500/month to use however you’d like * Complimentary lunch and snacks for in-office employees * One Medical membership, and more! Show more [Apply now >](https://jobicy.com/jobs/160991-machine-learning-engineer-core-evaluations.md) > Annual salary information is not provided for this position. Explore salary ranges for similar roles in our [Salary Directory ›](https://jobicy.com/salaries.md) * ![Upload CV](data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI2NSIgaGVpZ2h0PSI2NSIgZmlsbD0ibm9uZSIgeG1sbnM6dj0iaHR0cHM6Ly92ZWN0YS5pby9uYW5vIj48ZyBjbGlwLXBhdGg9InVybCgjQSkiPjxwYXRoIGQ9Ik0wIDBINjVWNjVIMFYwWiIgZmlsbD0iIzAyOWFlYiIvPjxnIGZpbGw9IiNmZmYiIHN0cm9rZT0iI2ZmZiIgc3Ryb2tlLXdpZHRoPSIyIj48cGF0aCBkPSJNMzMuMDQ5IDE1LjQ1NGExLjQzIDEuNDMgMCAwIDAtMi4wOTcgMGwtNy41NzkgOC4xNDdhMS4zOCAxLjM4IDAgMCAwIC4wOSAxLjk3MyAxLjQ0IDEuNDQgMCAwIDAgMi4wMDgtLjA4OGw1LjEwOS01LjQ5MnYyMC42MWExLjQxIDEuNDEgMCAwIDAgMS40MjEgMS4zOTdjLjc4NSAwIDEuNDIxLS42MjUgMS40MjEtMS4zOTd2LTIwLjYxbDUuMTA5IDUuNDkyYTEuNDQgMS40NCAwIDAgMCAyLjAwOC4wODggMS4zOCAxLjM4IDAgMCAwIC4wOS0xLjk3M2wtNy41NzktOC4xNDZ6TTE2Ljc2OSAzOC40YzAtLjc3My0uNjItMS40LTEuMzg1LTEuNFMxNCAzNy42MjcgMTQgMzguNHYuMTAybC4yMTUgNi4yMjljLjIyMyAxLjY4LjcwMSAzLjA5NSAxLjgxMyA0LjIxOHMyLjUxIDEuNjA3IDQuMTcyIDEuODMzYzEuNi4yMTggMy42MzYuMjE4IDYuMTYuMjE4aDExLjI4bDYuMTYtLjIxOGMxLjY2Mi0uMjI2IDMuMDYxLS43MDkgNC4xNzItMS44MzNzMS41ODktMi41MzggMS44MTMtNC4yMThDNTAgNDMuMTEzIDUwIDQxLjA1NSA1MCAzOC41MDNWMzguNGMwLS43NzMtLjYyLTEuNC0xLjM4NS0xLjRzLTEuMzg1LjYyNy0xLjM4NSAxLjRsLS4xOSA1Ljk1OGMtLjE4MiAxLjM3LS41MTUgMi4wOTUtMS4wMjYgMi42MTJzLTEuMjI4Ljg1My0yLjU4MyAxLjAzOGMtMS4zOTUuMTktMy4yNDMuMTkzLTUuODkzLjE5M0gyNi40NjJjLTIuNjUgMC00LjQ5OC0uMDAzLTUuODkzLS4xOTMtMS4zNTUtLjE4NC0yLjA3Mi0uNTIxLTIuNTgzLTEuMDM4cy0uODQ0LTEuMjQyLTEuMDI2LTIuNjEyYy0uMTg3LTEuNDEtLjE5MS0zLjI3OS0uMTkxLTUuOTU4eiIvPjwvZz48L2c+PGRlZnM+PGNsaXBQYXRoIGlkPSJBIj48cGF0aCBmaWxsPSIjZmZmIiBkPSJNMCAwaDY1djY1SDB6Ii8+PC9jbGlwUGF0aD48L2RlZnM+PC9zdmc+) ### Upload your resume now To unlock remote work opportunities and be discovered by global employers. This job listing has been manually reviewed by the Jobicy Trust & Safety Team for compliance with our posting guidelines, including verification of the company's legitimacy, accuracy of job details, clarity of remote work policy, and absence of misleading or fraudulent content. ## Role DNA A quick view of the complexity, pace, ownership and collaboration implied by the job description. ### Job Complexity 5/5EasyHard ### Pace & Pressure 4/5RelaxedFast-paced ### Autonomy Level 5/5GuidedFull ownership ### Communication Load 5/5IndependentCollaborative AI insightThis is a founding, technically deep role requiring expertise across speech ML, evaluation methodology, statistics, user research, and production engineering. The candidate must independently define evaluation strategy while influencing several technical and product teams. ## Salary analysis Estimated compensation compared with the broader US market for similar roles. Estimated job median (per year)Highly competitive$235,000US market range (per year)$190k–$280k0$308k AI insightThe yearly pay period is inferred from the job context; it was not explicitly stated by the employer. No employer base-pay amount was disclosed. The annual USD figures are AI-estimated market compensation for a senior/founding speech ML evaluation engineer in the US market; the yearly cadence is inferred from the full-time professional role. ## Core skills Skills and capabilities most closely associated with this opportunity. [Machine Learning](https://jobicy.com/jobs?search_keywords=Machine%20Learning.md)[Speech Recognition](https://jobicy.com/jobs?search_keywords=Speech%20Recognition.md)[Text-to-Speech](https://jobicy.com/jobs?search_keywords=Text-to-Speech.md)[ASR](https://jobicy.com/jobs?search_keywords=ASR.md)[TTS](https://jobicy.com/jobs?search_keywords=TTS.md)[Model Evaluation](https://jobicy.com/jobs?search_keywords=Model%20Evaluation.md)[Statistical Analysis](https://jobicy.com/jobs?search_keywords=Statistical%20Analysis.md)[User Studies](https://jobicy.com/jobs?search_keywords=User%20Studies.md)[A/B Testing](https://jobicy.com/jobs?search_keywords=AB%20Testing.md)[Python](https://jobicy.com/jobs?search_keywords=Python.md)[Model Fine-Tuning](https://jobicy.com/jobs?search_keywords=Model%20Fine-Tuning.md)[Evaluation Pipelines](https://jobicy.com/jobs?search_keywords=Evaluation%20Pipelines.md)[Data Visualization](https://jobicy.com/jobs?search_keywords=Data%20Visualization.md)[Dashboard Development](https://jobicy.com/jobs?search_keywords=Dashboard%20Development.md)[Large-Scale Machine Learning](https://jobicy.com/jobs?search_keywords=Large-Scale%20Machine%20Learning.md) Sample interview questionsHow would you design an evaluation framework for a new text-to-speech model?I would start by translating product and research goals into objective and subjective metrics, such as intelligibility, speaker similarity, prosody, latency, and perceived naturalness. I would build a representative test suite, establish reproducible inference conditions, use statistical confidence intervals, and supplement automated metrics with blinded listener studies. What are the limitations of subjective audio evaluations, and how would you reduce bias? Listener preferences can vary because of task framing, audio equipment, language background, fatigue, and inconsistent rater quality. I would use randomized and blinded samples, clear rubrics, qualification tests, quality-control items, balanced assignments, and sufficient sample sizes with predefined statistical analysis. Describe how you would compare two ASR systems when their aggregate word error rates are similar. I would segment results by speaker demographics, noise conditions, accents, domains, utterance length, and vocabulary type to identify where the models differ. I would also inspect substitution, deletion, and insertion patterns, evaluate latency and robustness, and use paired statistical tests to determine whether observed differences are meaningful. How would you turn a product requirement such as more natural conversational speech into measurable metrics? I would work with product and research stakeholders to define the specific qualities behind naturalness, such as turn-taking timing, prosody, emotional consistency, pronunciation, and interruption handling. I would then pair targeted automated proxies with structured human-evaluation questions and validate that the metrics correlate with user preference and product outcomes. What would your first 90 days look like as the founding member of an ML evaluation team? I would first map existing models, datasets, release processes, and quality risks with the model, data, and product teams. Next, I would prioritize a minimum viable evaluation suite and reporting dashboard for the highest-impact models, establish experiment and dataset versioning, and propose a roadmap for scaling user studies, automated regression testing, and team capabilities. Next step ## Apply now. Follow the employer’s application method and review Jobicy’s safety guidance before sharing personal information. Keep exploring ## Related remote jobs. * ![Deepgram logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2020/12/WRILS-201212160008-406229.jpg) Deepgram Nov 10 New ### [Director of Research, Text-to-Speech](https://jobicy.com/jobs/161053-director-of-research-text-to-speech.md) Company Overview Deepgram is bringing the world’s most natural interface, voice, to the technology we use every day. Our speech models and real-time infrastructure power AI systems that keep up… * ![Upstart logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2025/06/bc2cfa2b-221-1.jpeg) Upstart Nov 10 New ### [Applied Scientist](https://jobicy.com/jobs/161042-applied-scientist-3.md) About Upstart At Upstart, we’re united by a mission that matters: to radically reduce the cost and complexity of borrowing for all Americans. Every day, we bring creativity, experimentation, and… * ![Quilter logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2025/08/138e7ef9-221.png) Quilter Nov 10 New ### [Applied Scientist](https://jobicy.com/jobs/161034-applied-scientist-2.md) About Quilter At Quilter, we are helping electrical engineers save time and accomplish more by automating the tedious and time-consuming task of designing printed circuit boards (PCBs). Our small team… * ![Cantina Labs logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2026/11/e52bb5e84323-221.webp) Cantina Labs Nov 10 New ### [Research Scientist, Real-Time Video Generation](https://jobicy.com/jobs/161001-research-scientist-real-time-video-generation.md) About Cantina Cantina Labs is a social AI company, developing a suite of advanced real-time models that push the boundaries of expression, personality, and realism. We bring characters to life,… * ![Cantina Labs logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2026/11/e52bb5e84323-221.webp) Cantina Labs Nov 10 New ### [Machine Learning Engineer, Voice Conversion](https://jobicy.com/jobs/160999-machine-learning-engineer-voice-conversion.md) About Cantina: Cantina is a new social platform founded by Sean Parker with the most advanced AI character creator. Our bots are lifelike, social creatures that can interact wherever people… * ![Welo Global logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2026/06/83c40f94-221.webp) Welo Global Nov 10 New ### [Project Lyra Danish Culture Expert](https://jobicy.com/jobs/160998-project-lyra-danish-culture-expert.md) Help shape how AI understands and speaks Danish. We’re launching a short-term project that needs Danish speakers to review, rewrite, and create Danish text that trains AI language models. You’ll… * ![Cantina Labs logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2026/11/e52bb5e84323-221.webp) Cantina Labs Nov 10 New ### [Machine Learning Engineer, TTS](https://jobicy.com/jobs/160994-machine-learning-engineer-tts.md) About Cantina Cantina is a new social platform founded by Sean Parker with the most advanced AI character creator. Our bots are lifelike, social creatures that can interact wherever people… * ![DuckDuckGo logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2021/11/ab8415dd5798a360323ce06beaf30c35.png) DuckDuckGo Nov 10 New ### [Senior Data Scientist](https://jobicy.com/jobs/154962-senior-data-scientist-7.md) Who We Are Hi, we’re DuckDuckGo, the online protection company and remote-first team of 300+ on a mission to raise the standard of trust online. Founded in 2008 and profitable… * ![Bayesian Health logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2025/08/f74593d8-221.png) Bayesian Health Nov 10 New ### [Senior/Staff Machine Learning Data Scientist](https://jobicy.com/jobs/152824-senior-staff-machine-learning-data-scientist.md) Senior/Staff Machine Learning Data Scientist In Brief We’re an early-stage startup on a mission to make healthcare proactive by empowering physicians, nurses, and care team members with real-time data to… * ![Beyond Finance logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2022/01/68cd015e123539d873279c7b82f1ca6a.jpg) Beyond Finance Nov 10 New ### [Manager, Strategy & Analytics](https://jobicy.com/jobs/152802-manager-strategy-analytics.md) At Beyond Finance, we’ve made it our mission to help everyday Americans escape the endless cycle of crippling debt and step into a brighter financial future. Through compassionate, individualized care,… [Browse all jobs](https://jobicy.com/jobs.md)