[All remote jobs](https://jobicy.com/jobs.md)[![Deepgram logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2020/12/WRILS-201212160008-406229.jpg)](https://jobicy.com/company/deepgram.md)[Deepgram](https://jobicy.com/company/deepgram.md) # Embedded AI Engineer, On-Device Models AI Summary Deepgram is hiring a senior-to-staff Embedded AI Engineer to make speech models performant on constrained edge hardware. The role focuses on custom kernels, low-level runtime components, quantization, operator fusion, and architecture-specific optimization for SoCs, NPUs, DSPs, and embedded targets. The engineer will benchmark latency, accuracy, memory, power, and thermals while extending vendor toolchains where standard operators are insufficient. Success requires close collaboration with applied ML teams and silicon vendors, plus the ability to independently turn ambiguous hardware constraints into reusable production components. This is a highly specialized, rapidly evolving role in on-device speech AI.[Apply for this job](#job-application)[View company](https://jobicy.com/company/deepgram.md)ShareRemote from[USA](https://jobicy.com/job-region/usa.md)SalaryUSD 219,300–274,100 / yrDepartment[Software Engineering](https://jobicy.com/categories/engineering.md)EmploymentFull TimeExperienceSeniorPublished7 Nov 2026Apply before7 Dec 2026Listing views54Application actions4Application toolkit ## Make your next move. Prepare your resume, explore your fit, and draft a cover letter for this opportunity. Opportunity details ## About this role. ### Company Overview [Deepgram](https://jobicy.com/company/deepgram.md) is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing real-time APIs for speech-to-text (STT), text-to-speech (TTS), and building production-grade voice agents at scale. More than 200,000 developers and 1,300+ organizations build voice offerings that are ‘Powered by Deepgram’, including Twilio, Cloudflare, Sierra, Decagon, Vapi, Daily, Cresta, Granola, and Jack in the Box. Deepgram’s voice-native foundation models are accessed through cloud APIs or as self-hosted and on-premises software, with unmatched accuracy, low latency, and cost efficiency. Backed by a recent Series C led by leading global investors and strategic partners, Deepgram has processed over 50,000 years of audio and transcribed more than 1 trillion words. There is no organization in the world that understands voice better than Deepgram. ### Company Operating Rhythm At Deepgram, we expect an AI-first mindset—AI use and comfort aren’t optional, they’re core to how we operate, innovate, and measure performance. Every team member who works at Deepgram is expected to actively use and experiment with advanced AI tools, and even build your own into your everyday work. We measure how effectively AI is applied to deliver results, and consistent, creative use of the latest AI capabilities is key to success here. Candidates should be comfortable adopting new models and modes quickly, integrating AI into their workflows, and continuously pushing the boundaries of what these technologies can do. Additionally, we move at the pace of AI. Change is rapid, and you can expect your day-to-day work to evolve just as quickly. This may not be the right role if you’re not excited to experiment, adapt, think on your feet, and learn constantly, or if you’re seeking something highly prescriptive with a traditional 9-to-5. ### About the Role Deepgram’s speech models are among the fastest and most accurate in the world, and we have deep machinery for running them on NVIDIA GPUs. Our customers need them on everything else: non-NVIDIA accelerators, embedded SoCs, mobile application processors, DSPs and NPUs, and purpose-built devices with tight memory, compute, thermal, and power budgets. When a target platform’s standard kernels and runtime can’t run a Deepgram model well enough, someone has to go below them. That is this role. As an Embedded AI Engineer on the Partner Platform Engineering team, you work at the lowest layer of our edge stack. You write and optimize custom kernels and operators for specific hardware, collapse models onto device-specific execution units, and do the target-side quantization and assembly-level tuning that standard toolchains can’t. You hand what you build up to Applied ML Engineers, who fit Deepgram models to your kernels. Your work is what makes a new hardware platform viable for Deepgram at all. This role is a great fit for a senior embedded engineer who has spent their career close to the metal and wants to point that at speech AI, or a staff-level engineer who wants to define how Deepgram’s models get onto new silicon. We’ll set the level to your experience. ### What You’ll Do * Write and optimize custom kernels and operators (C, C++, Rust, and platform assembly or intrinsics) for non-NVIDIA accelerators, embedded SoCs, DSPs, and NPUs where the vendor’s standard operator set is insufficient for Deepgram models. * Own target-side optimization: collapse models onto device execution units through quantization, operator fusion, memory layout, and architecture-specific compilation to meet latency, memory, power, and thermal budgets. * Integrate with vendor NPU/DSP toolchains and edge inference runtimes, and extend them with custom operators when the graph doesn’t map cleanly. * Deliver kernels and runtime components as reusable building blocks that Applied ML Engineers can target when adapting models, with clear interfaces and documented constraints. * Build performance-critical runtime code for embedded environments, including embedded Linux, bare-metal, and RTOS targets. * Establish per-platform benchmarking and validation for latency, accuracy, power, memory footprint, and utilization, and catch regressions before they ship. * Partner with silicon and platform vendors on SDK integration and low-level performance tuning for new chipsets and reference platforms. * Feed hardware constraints back to Applied ML and Research so model designs are easier to land on constrained targets. ### You’ll Love This Role If You * Find deep satisfaction in making a large model run on hardware that was never meant to run it, and still hitting accuracy and latency targets. * Reach for the profiler and the ISA manual before you reach for a bigger chip. * Would rather write the kernel than wait for the vendor to ship it. * Care about the details that don’t show up in a cloud benchmark: cold start, power draw, thermals, memory fragmentation, cache behavior. * Prefer hard, constrained, ship-it problems over open-ended research. * Care about the details that don’t show up in a cloud benchmark: cold-start time, power draw, thermals, and memory fragmentation. ### It’s Important To Us That You Have * Experience delivering production systems on resource-constrained hardware — embedded systems, mobile, edge AI, or small low-power devices. * Strong proficiency in C, C++, and/or Rust, with experience writing performance-critical code for constrained environments. * Hands-on experience with model optimization for on-device deployment, including quantization, pruning, knowledge distillation, or architecture-specific compilation. * Familiarity with edge inference runtimes (e.g., ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains. * A strong understanding of hardware-software interaction — CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management — and how they affect inference performance. * Experience working close to the metal: bare-metal or RTOS environments (e.g., FreeRTOS, Zephyr), embedded Linux, or microcontroller and edge SoC development. * Strong communication skills and a builder mindset — you can scope an ambiguous optimization problem, drive it to a measurable result, and explain the tradeoffs clearly. ### It Would Be Great if You Had * Experience with real-time audio processing on embedded platforms — DSP pipelines, audio codec optimization, wake-word or always-on listening, or streaming inference on microcontrollers and edge SoCs. * Depth in ML optimization techniques — custom quantization schemes, mixed-precision inference, or neural architecture search for edge targets. * Background in hardware evaluation and benchmarking — systematically comparing accelerators, SoCs, or GPUs for specific workload profiles. * Experience shipping AI features in consumer products at scale, and the instinct for what “production quality” means on a battery-powered device. * Familiarity with model compilation and optimization toolchains and their tradeoffs across hardware targets. * Experience with secure, robust on-device deployment practices — code signing, encrypted model storage, and safe update mechanisms. Notice: We’re aware of individuals impersonating Deepgram recruiters. All legitimate Deepgram recruiting communication comes from an @[deepgram.com](http://deepgram.com) email address. If you’ve received a message claiming to be Deepgram, please forward it to careers@deepgram.com. Show more [Apply now >](https://jobicy.com/jobs/159501-embedded-ai-engineer-on-device-models.md) * ![Upload CV](data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI2NSIgaGVpZ2h0PSI2NSIgZmlsbD0ibm9uZSIgeG1sbnM6dj0iaHR0cHM6Ly92ZWN0YS5pby9uYW5vIj48ZyBjbGlwLXBhdGg9InVybCgjQSkiPjxwYXRoIGQ9Ik0wIDBINjVWNjVIMFYwWiIgZmlsbD0iIzAyOWFlYiIvPjxnIGZpbGw9IiNmZmYiIHN0cm9rZT0iI2ZmZiIgc3Ryb2tlLXdpZHRoPSIyIj48cGF0aCBkPSJNMzMuMDQ5IDE1LjQ1NGExLjQzIDEuNDMgMCAwIDAtMi4wOTcgMGwtNy41NzkgOC4xNDdhMS4zOCAxLjM4IDAgMCAwIC4wOSAxLjk3MyAxLjQ0IDEuNDQgMCAwIDAgMi4wMDgtLjA4OGw1LjEwOS01LjQ5MnYyMC42MWExLjQxIDEuNDEgMCAwIDAgMS40MjEgMS4zOTdjLjc4NSAwIDEuNDIxLS42MjUgMS40MjEtMS4zOTd2LTIwLjYxbDUuMTA5IDUuNDkyYTEuNDQgMS40NCAwIDAgMCAyLjAwOC4wODggMS4zOCAxLjM4IDAgMCAwIC4wOS0xLjk3M2wtNy41NzktOC4xNDZ6TTE2Ljc2OSAzOC40YzAtLjc3My0uNjItMS40LTEuMzg1LTEuNFMxNCAzNy42MjcgMTQgMzguNHYuMTAybC4yMTUgNi4yMjljLjIyMyAxLjY4LjcwMSAzLjA5NSAxLjgxMyA0LjIxOHMyLjUxIDEuNjA3IDQuMTcyIDEuODMzYzEuNi4yMTggMy42MzYuMjE4IDYuMTYuMjE4aDExLjI4bDYuMTYtLjIxOGMxLjY2Mi0uMjI2IDMuMDYxLS43MDkgNC4xNzItMS44MzNzMS41ODktMi41MzggMS44MTMtNC4yMThDNTAgNDMuMTEzIDUwIDQxLjA1NSA1MCAzOC41MDNWMzguNGMwLS43NzMtLjYyLTEuNC0xLjM4NS0xLjRzLTEuMzg1LjYyNy0xLjM4NSAxLjRsLS4xOSA1Ljk1OGMtLjE4MiAxLjM3LS41MTUgMi4wOTUtMS4wMjYgMi42MTJzLTEuMjI4Ljg1My0yLjU4MyAxLjAzOGMtMS4zOTUuMTktMy4yNDMuMTkzLTUuODkzLjE5M0gyNi40NjJjLTIuNjUgMC00LjQ5OC0uMDAzLTUuODkzLS4xOTMtMS4zNTUtLjE4NC0yLjA3Mi0uNTIxLTIuNTgzLTEuMDM4cy0uODQ0LTEuMjQyLTEuMDI2LTIuNjEyYy0uMTg3LTEuNDEtLjE5MS0zLjI3OS0uMTkxLTUuOTU4eiIvPjwvZz48L2c+PGRlZnM+PGNsaXBQYXRoIGlkPSJBIj48cGF0aCBmaWxsPSIjZmZmIiBkPSJNMCAwaDY1djY1SDB6Ii8+PC9jbGlwUGF0aD48L2RlZnM+PC9zdmc+) ### Upload your resume now To unlock remote work opportunities and be discovered by global employers. This job listing has been manually reviewed by the Jobicy Trust & Safety Team for compliance with our posting guidelines, including verification of the company's legitimacy, accuracy of job details, clarity of remote work policy, and absence of misleading or fraudulent content. ## Role DNA A quick view of the complexity, pace, ownership and collaboration implied by the job description. ### Job Complexity 5/5EasyHard ### Pace & Pressure 5/5RelaxedFast-paced ### Autonomy Level 5/5GuidedFull ownership ### Communication Load 4/5IndependentCollaborative AI insightThe position requires deep expertise across embedded programming, hardware architecture, compiler/runtime optimization, and on-device ML inference. Engineers must solve novel performance problems under strict latency, memory, thermal, and power constraints. ## Salary analysis Estimated compensation compared with the broader US market for similar roles. Estimated job median (per year)Market rate$246,700US market range (per year)$210k–$290k0$319k AI insightThe employer discloses a yearly base-pay range of $219,300–$274,100 USD, with a midpoint of $246,700. This is a credible range for a senior or staff-level embedded AI engineering role in the US; the estimated US market range is $210,000–$290,000 per year. ## Core skills Skills and capabilities most closely associated with this opportunity. [Embedded systems](https://jobicy.com/jobs?search_keywords=Embedded%20systems.md)[C programming](https://jobicy.com/jobs?search_keywords=C%20programming.md)[C++ programming](https://jobicy.com/jobs?search_keywords=C%2B%2B%20programming.md)[Rust](https://jobicy.com/jobs?search_keywords=Rust.md)[Custom kernel optimization](https://jobicy.com/jobs?search_keywords=Custom%20kernel%20optimization.md)[Model quantization](https://jobicy.com/jobs?search_keywords=Model%20quantization.md)[Operator fusion](https://jobicy.com/jobs?search_keywords=Operator%20fusion.md)[Edge AI inference](https://jobicy.com/jobs?search_keywords=Edge%20AI%20inference.md)[NPU and DSP toolchains](https://jobicy.com/jobs?search_keywords=NPU%20and%20DSP%20toolchains.md)[ONNX Runtime](https://jobicy.com/jobs?search_keywords=ONNX%20Runtime.md)[TensorRT](https://jobicy.com/jobs?search_keywords=TensorRT.md)[TensorFlow Lite](https://jobicy.com/jobs?search_keywords=TensorFlow%20Lite.md)[RTOS development](https://jobicy.com/jobs?search_keywords=RTOS%20development.md)[Embedded Linux](https://jobicy.com/jobs?search_keywords=Embedded%20Linux.md)[Performance benchmarking](https://jobicy.com/jobs?search_keywords=Performance%20benchmarking.md) Sample interview questionsDescribe a time you optimized an ML workload for a device with tight latency, memory, or power constraints.I begin by establishing a measured baseline using device-side profiling, then identify the dominant bottleneck in compute, memory bandwidth, scheduling, or data movement. I apply targeted changes such as quantization, operator fusion, layout changes, or platform-specific kernels, and validate accuracy alongside latency, memory, and power metrics. How would you approach implementing a custom operator when an NPU runtime does not efficiently support part of a speech model? I would first verify that graph rewriting or supported operator decompositions cannot meet the target metrics. Then I would define a stable operator interface, implement and tune the hardware-specific kernel, integrate it with the runtime’s execution and memory model, and benchmark it across representative inputs and device conditions. What factors do you consider when selecting an integer or mixed-precision quantization strategy? I consider operator sensitivity, activation distributions, available hardware data paths, calibration quality, memory constraints, and end-to-end accuracy requirements. I validate per-layer error and production workload quality, using mixed precision for sensitive operations when uniform low-bit quantization produces unacceptable degradation. How do you prevent performance regressions across multiple embedded hardware targets? I create repeatable per-platform benchmark suites with fixed model versions, representative audio inputs, and thresholds for latency, memory, accuracy, power, and cold-start behavior. Results should run in CI or a controlled hardware lab, with regressions attributed to model, compiler, runtime, or kernel changes before release. How would you communicate a hardware limitation to an Applied ML team? I would provide measurable constraints, such as unsupported operations, memory-bandwidth limits, or latency budgets, along with profiling evidence. I would propose practical alternatives—such as operator substitutions, fusion opportunities, quantization constraints, or architecture changes—and explain their expected quality and performance tradeoffs. Next step ## Apply now. Follow the employer’s application method and review Jobicy’s safety guidance before sharing personal information. Keep exploring ## Related remote jobs. * ![Ashby logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2025/08/63864ee6-221.png) Ashby Nov 7 New ### [Demo Engineer](https://jobicy.com/jobs/159568-demo-engineer-3.md) About this role As Ashby’s first Demo Engineer, you’ll own the systems that make our demo environments reliable, current, and tailored — the infrastructure the entire go-to-market team leans on… * ![Welo Global logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2026/06/83c40f94-221.webp) Welo Global Nov 7 New ### [Maps Personalization Relevance Rater – English (Australia)](https://jobicy.com/jobs/159539-maps-personalization-relevance-rater-english-australia.md) We are looking for qualified raters to support a project focused on evaluating personalized search and location recommendations. This role involves rating the relevance and usefulness of suggested places based… * ![Welo Global logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2026/06/83c40f94-221.webp) Welo Global Nov 7 New ### [Maps Personalization Relevance Rater – Chinese (Hong Kong)](https://jobicy.com/jobs/159538-maps-personalization-relevance-rater-chinese-hong-kong.md) Contract type: Freelance Language: Chinese (Hong Kong) Estimated volume: 8 – 10 hours Start date: The project runs on a weekly basis. We are looking for qualified raters to support… * ![Welo Global logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2026/06/83c40f94-221.webp) Welo Global Nov 7 New ### [Maps Personalization Quality Reviewer – Japanese (Japan)](https://jobicy.com/jobs/159537-maps-personalization-quality-reviewer-japanese-japan.md) We’re seeking an experienced Quality Reviewer to audit work completed by raters evaluating a personalization feature within a maps/search experience. As a Reviewer, you will ensure outputs meet formatting and… * ![Welo Global logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2026/06/83c40f94-221.webp) Welo Global Nov 7 New ### [Korean Metadata Specialist in Japan](https://jobicy.com/jobs/159536-korean-metadata-specialist-in-japan.md) Welocalize is currently seeking a Korean Metadata Specialist for a full-time position in Japan. You will be working remotely as a vendor for a leading consumer technology company, contributing to… * ![Luxury Presence logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2022/05/f2b241fb769140fff2f70e8efaf5c0ff.jpg) Luxury Presence Nov 7 New ### [Junior Data Specialist](https://jobicy.com/jobs/159530-junior-data-specialist.md) Luxury Presence is building the AI growth platform for real estate. Backed by Bessemer Venture Partners and other top investors, we’re a Series C company that has hit $100M in… * ![EverAI logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2025/06/91735b0b-221.png) EverAI Nov 7 New ### [Mid/Senior AI LLM Engineer (Full Remote – Spain)](https://jobicy.com/jobs/159526-mid-senior-ai-llm-engineer-full-remote-spain.md) Our Vision & Products 🚀 EverAI — Building the Future of AI Companionship One of the Top 15 Largest & Fastest-Growing AI Companies in the World 50 Million Users in… * ![Masabi logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2025/06/f2355465-221.png) Masabi Nov 7 New ### [Software Engineer – Data team](https://jobicy.com/jobs/159520-software-engineer-data-team.md) About Us_// At Masabi, we’re driving the fare payment revolution, powering the journeys of millions all over the world. We build fare collection platforms that allow riders to seamlessly buy… * ![ecosio logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2025/06/eec17a98-221.png) ecosio Nov 7 New ### [Quality Assurance Engineer (m/f/d)](https://jobicy.com/jobs/159518-quality-assurance-engineer-m-f-d.md) ecosio is a fast-growing, innovative service company and a leading provider of B2B integration, specialising in electronic data interchange (EDI), Web EDI and e-invoicing. ecosio is part of Vertex, Inc.,… * ![Databricks logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2021/12/4e3f864ba9cf3c62d471e1c62414d098.jpg) Databricks Nov 7 New ### [Sr. Solutions Architect – Travel and Hospitality](https://jobicy.com/jobs/159512-sr-solutions-architect-travel-and-hospitality.md) Sr. Solutions Architect (L6) The Role As a Sr. Solutions Architect, you are an expert-level technical leader responsible for the organization’s most complex customer engagements. You drive consumption strategy, own… [Browse all jobs](https://jobicy.com/jobs.md)