[All remote jobs](https://jobicy.com/jobs.md)[![Deepgram logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2020/12/WRILS-201212160008-406229.jpg)](https://jobicy.com/company/deepgram.md)[Deepgram](https://jobicy.com/company/deepgram.md) # Backend Engineer- Inference Services Review the role, location requirements, compensation details, and application process before deciding whether this opportunity fits your next career move. [Apply for this job](#job-application)[View company](https://jobicy.com/company/deepgram.md)ShareRemote from[USA](https://jobicy.com/job-region/usa.md)SalaryUSD 150k–220k / yrDepartment[Software Engineering](https://jobicy.com/categories/engineering.md)EmploymentFull TimeExperienceSeniorPublished29 Oct 2026Apply before28 Nov 2026Listing views47Application actions3Application toolkit ## Make your next move. Prepare your resume, explore your fit, and draft a cover letter for this opportunity. AI Summary ## The role, at a glance. Deepgram is hiring a Backend Engineer to build and improve inference services for speech-processing products. The role focuses on secure, scalable distributed services, networking, audio transcoding, compute orchestration, scheduling, and latency and memory optimization. Candidates need at least three years of industry experience, strong Rust, C, or C++ skills, Python proficiency, Git, and UNIX systems expertise. This is a high-impact, fast-moving engineering position that requires frequent experimentation with AI tools and close collaboration with Product. Experience with machine learning frameworks, transformer architectures, and audio processing is a strong advantage. ## Role DNA A quick view of the complexity, pace, ownership and collaboration implied by the job description. ### Job Complexity 5/5EasyHard ### Pace & Pressure 5/5RelaxedFast-paced ### Autonomy Level 4/5GuidedFull ownership ### Communication Load 4/5IndependentCollaborative AI insightThe role involves diagnosing complex interactions among networking, scheduling, high-performance computing, and real-time speech inference systems. It also demands systems-level programming expertise and rapid technical decisions in a fast-changing AI environment. ## Salary analysis Estimated compensation compared with the broader US market for similar roles. Estimated job median (per year)Highly competitive$185,000US market range (per year)$145k–$220k0$242.00k AI insightThe employer discloses an annual base compensation range of $150,000–$220,000 USD, with a midpoint of $185,000. This is a credible range for a U.S.-remote backend engineer specializing in high-performance inference and distributed systems; the estimated U.S. market range is $145,000–$220,000 annually. ## Core skills Skills and capabilities most closely associated with this opportunity. [Rust](https://jobicy.com/jobs?search_keywords=Rust.md)[C++](https://jobicy.com/jobs?search_keywords=C%2B%2B.md)[Python](https://jobicy.com/jobs?search_keywords=Python.md)[Distributed systems](https://jobicy.com/jobs?search_keywords=Distributed%20systems.md)[High-performance computing](https://jobicy.com/jobs?search_keywords=High-performance%20computing.md)[Networking](https://jobicy.com/jobs?search_keywords=Networking.md)[Speech processing](https://jobicy.com/jobs?search_keywords=Speech%20processing.md)[Audio transcoding](https://jobicy.com/jobs?search_keywords=Audio%20transcoding.md)[Performance optimization](https://jobicy.com/jobs?search_keywords=Performance%20optimization.md)[UNIX systems](https://jobicy.com/jobs?search_keywords=UNIX%20systems.md)[Git](https://jobicy.com/jobs?search_keywords=Git.md)[Machine learning](https://jobicy.com/jobs?search_keywords=Machine%20learning.md)[PyTorch](https://jobicy.com/jobs?search_keywords=PyTorch.md)[System debugging](https://jobicy.com/jobs?search_keywords=System%20debugging.md) Sample interview questionsHow would you investigate a sudden increase in p99 latency in a real-time speech inference service?I would first validate the metric and segment it by model, endpoint, region, request size, and deployment version. I would correlate latency with CPU, GPU, memory, network, queue depth, scheduling, and downstream dependency metrics, then use tracing and profiling to isolate the slow path. After reproducing the issue where possible, I would mitigate the highest-impact bottleneck and add monitoring or regression tests to prevent recurrence. Describe how you would design a scalable service for orchestrating distributed inference workloads. I would separate request admission, scheduling, execution, and result delivery into clearly defined components with observable interfaces. The scheduler would account for resource availability, model affinity, queue priority, deadlines, and backpressure, while workers would be designed for safe retries and graceful failure handling. I would also build in capacity metrics, load shedding, and horizontal scaling so the system remains reliable under variable demand. What are important considerations when using Rust or C++ for a low-latency backend service? Key considerations include memory ownership and allocation behavior, concurrency safety, efficient serialization, connection management, and minimizing unnecessary copies. I would profile before optimizing, focus on the critical path, and use bounded queues and backpressure to avoid unbounded resource use. Strong testing, sanitizers, structured logging, and safe rollout practices are also essential for maintaining reliability in systems-level services. How would you approach optimizing audio transcoding or speech-processing throughput without reducing quality? I would establish quality and latency baselines, profile the pipeline, and identify costly decode, encode, conversion, and data-copy operations. Possible improvements include batching where latency budgets permit, using streaming processing, selecting efficient codecs or sample-rate conversion paths, and exploiting hardware acceleration when appropriate. Every change would be validated with representative audio, load tests, and quality metrics to ensure performance gains do not harm output quality. How do you decide between a minimal production fix and a more comprehensive redesign? I assess customer impact, reliability risk, time sensitivity, technical debt, and the likelihood that the problem will recur. For an urgent and well-understood issue, I prefer a narrowly scoped, observable fix with a rollback plan, followed by a documented plan for deeper improvements. If the existing design creates systemic risk or blocks future work, I align stakeholders on a staged redesign with milestones and measurable outcomes. Opportunity details ## About this role. ### Company Overview Deepgram is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing real-time APIs for speech-to-text (STT), text-to-speech (TTS), and building production-grade voice agents at scale. More than 200,000 developers and 1,300+ organizations build voice offerings that are ‘Powered by Deepgram’, including Twilio, Cloudflare, Sierra, Decagon, Vapi, Daily, Cresta, Granola, and Jack in the Box. Deepgram’s voice-native foundation models are accessed through cloud APIs or as self-hosted and on-premises software, with unmatched accuracy, low latency, and cost efficiency. Backed by a recent Series C led by leading global investors and strategic partners, Deepgram has processed over 50,000 years of audio and transcribed more than 1 trillion words. There is no organization in the world that understands voice better than Deepgram. ### Company Operating Rhythm At Deepgram, we expect an AI-first mindset—AI use and comfort aren’t optional, they’re core to how we operate, innovate, and measure performance. Every team member who works at Deepgram is expected to actively use and experiment with advanced AI tools, and even build your own into your everyday work. We measure how effectively AI is applied to deliver results, and consistent, creative use of the latest AI capabilities is key to success here. Candidates should be comfortable adopting new models and modes quickly, integrating AI into their workflows, and continuously pushing the boundaries of what these technologies can do. Additionally, we move at the pace of AI. Change is rapid, and you can expect your day-to-day work to evolve just as quickly. This may not be the right role if you’re not excited to experiment, adapt, think on your feet, and learn constantly, or if you’re seeking something highly prescriptive with a traditional 9-to-5. Opportunity: Deepgram is looking for a Backend Software Engineer to join the Engine team to lead the design and implementation of Deepgram’s products. You will design and implement secure, robust, and scalable services for speech processing; efficient, distributed compute orchestration; optimized scheduling, and more. Your skill at building highly reusable code that overcomes technical challenges is paired with an intuition for delightful user experiences. You will be a critical voice in Deepgram’s Product and Engineering teams, driving high impact products from start to finish. What You’ll Do * Improve Deepgram’s core inference services including areas in networking, speech processing, audio transcoding, and latency and memory optimization * Develop processes for measuring, building, and optimizing services to maximize system performance * Debug complex system issues that include networking, scheduling, and high performance computing interactions * Rapidly customize backend services to support our customer needs * Partner with Product to design and implement new services, features, and/or products end to end You’ll Love This Role If You * Thrive in a fast-paced, impact-driven environment where learning new skills on-the-fly is not only encouraged but a regular necessity * Enjoy balancing decisions about product and feature maturity to decide when to make minimally invasive changes versus when to incorporate detailed design work It’s Important To Us That You Have * 3+ years of experience in an industry role * Programming experience in Rust (or C, C++), with competence in Python * Excellent communication and organizational skills, both written and verbal. * A high level of experience and understanding of version control; preferably git. * Comprehensive experience with UNIX-style systems. It Would Be Great if You Had * Experience with modern machine learning, such as experience with a framework like Torch or implementation knowledge of architectures like CNNs, RNNS, and transformers * Experience with audio processing Notice: We’re aware of individuals impersonating Deepgram recruiters. All legitimate Deepgram recruiting communication comes from an @[deepgram.com](http://deepgram.com) email address. If you’ve received a message claiming to be Deepgram, please forward it to careers@deepgram.com. Show more [Apply now >](https://jobicy.com/jobs/157865-backend-engineer-inference-services.md) * ![Upload CV](data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI2NSIgaGVpZ2h0PSI2NSIgZmlsbD0ibm9uZSIgeG1sbnM6dj0iaHR0cHM6Ly92ZWN0YS5pby9uYW5vIj48ZyBjbGlwLXBhdGg9InVybCgjQSkiPjxwYXRoIGQ9Ik0wIDBINjVWNjVIMFYwWiIgZmlsbD0iIzAyOWFlYiIvPjxnIGZpbGw9IiNmZmYiIHN0cm9rZT0iI2ZmZiIgc3Ryb2tlLXdpZHRoPSIyIj48cGF0aCBkPSJNMzMuMDQ5IDE1LjQ1NGExLjQzIDEuNDMgMCAwIDAtMi4wOTcgMGwtNy41NzkgOC4xNDdhMS4zOCAxLjM4IDAgMCAwIC4wOSAxLjk3MyAxLjQ0IDEuNDQgMCAwIDAgMi4wMDgtLjA4OGw1LjEwOS01LjQ5MnYyMC42MWExLjQxIDEuNDEgMCAwIDAgMS40MjEgMS4zOTdjLjc4NSAwIDEuNDIxLS42MjUgMS40MjEtMS4zOTd2LTIwLjYxbDUuMTA5IDUuNDkyYTEuNDQgMS40NCAwIDAgMCAyLjAwOC4wODggMS4zOCAxLjM4IDAgMCAwIC4wOS0xLjk3M2wtNy41NzktOC4xNDZ6TTE2Ljc2OSAzOC40YzAtLjc3My0uNjItMS40LTEuMzg1LTEuNFMxNCAzNy42MjcgMTQgMzguNHYuMTAybC4yMTUgNi4yMjljLjIyMyAxLjY4LjcwMSAzLjA5NSAxLjgxMyA0LjIxOHMyLjUxIDEuNjA3IDQuMTcyIDEuODMzYzEuNi4yMTggMy42MzYuMjE4IDYuMTYuMjE4aDExLjI4bDYuMTYtLjIxOGMxLjY2Mi0uMjI2IDMuMDYxLS43MDkgNC4xNzItMS44MzNzMS41ODktMi41MzggMS44MTMtNC4yMThDNTAgNDMuMTEzIDUwIDQxLjA1NSA1MCAzOC41MDNWMzguNGMwLS43NzMtLjYyLTEuNC0xLjM4NS0xLjRzLTEuMzg1LjYyNy0xLjM4NSAxLjRsLS4xOSA1Ljk1OGMtLjE4MiAxLjM3LS41MTUgMi4wOTUtMS4wMjYgMi42MTJzLTEuMjI4Ljg1My0yLjU4MyAxLjAzOGMtMS4zOTUuMTktMy4yNDMuMTkzLTUuODkzLjE5M0gyNi40NjJjLTIuNjUgMC00LjQ5OC0uMDAzLTUuODkzLS4xOTMtMS4zNTUtLjE4NC0yLjA3Mi0uNTIxLTIuNTgzLTEuMDM4cy0uODQ0LTEuMjQyLTEuMDI2LTIuNjEyYy0uMTg3LTEuNDEtLjE5MS0zLjI3OS0uMTkxLTUuOTU4eiIvPjwvZz48L2c+PGRlZnM+PGNsaXBQYXRoIGlkPSJBIj48cGF0aCBmaWxsPSIjZmZmIiBkPSJNMCAwaDY1djY1SDB6Ii8+PC9jbGlwUGF0aD48L2RlZnM+PC9zdmc+) ### Upload your resume now To unlock remote work opportunities and be discovered by global employers. This job listing has been manually reviewed by the Jobicy Trust & Safety Team for compliance with our posting guidelines, including verification of the company's legitimacy, accuracy of job details, clarity of remote work policy, and absence of misleading or fraudulent content. Next step ## Apply now. Follow the employer’s application method and review Jobicy’s safety guidance before sharing personal information. Keep exploring ## Related remote jobs. * ![ClickUp logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2025/06/cd5be0ef-221.jpg) ClickUp Oct 29 New ### [Manager, Engineering – Product](https://jobicy.com/jobs/157911-manager-engineering-product.md) At ClickUp, we’re building the future of work: the first truly converged AI workspace unifying tasks, docs, chat, calendar, and enterprise search, all supercharged by context-driven AI. We are an… * ![Ping Identity logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2025/09/63e8d5a6-221.png) Ping Identity Oct 29 New ### [Software Quality Engineer](https://jobicy.com/jobs/157888-software-quality-engineer.md) About Ping Identity: At Ping Identity, we believe in making digital experiences both secure and seamless for all users, without compromise. We call this digital freedom. And it’s not just… * ![Ping Identity logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2025/09/63e8d5a6-221.png) Ping Identity Oct 29 New ### [Senior Mobile Developer](https://jobicy.com/jobs/157885-senior-mobile-developer.md) About Ping Identity: At Ping Identity, we believe in making digital experiences both secure and seamless for all users, without compromise. We call this digital freedom. And it’s not just… * ![GitLab logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2020/12/WRILS-201207055737-109952.jpg) GitLab Oct 29 New ### [Principal Engineer, Duo Agent Platform](https://jobicy.com/jobs/157884-principal-engineer-duo-agent-platform.md) GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital transformation. More than 50… * ![Dropbox logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2020/09/WRILS-200923071227-398427.jpg) Dropbox Oct 29 New ### [Full Stack Product Software Engineer](https://jobicy.com/jobs/157873-full-stack-product-software-engineer.md) Role Description As a Full Stack Engineer at Dropbox, you will help design and develop the seamless, scalable, and user-friendly experiences Dropbox users depend on. Your work will encompass both… * ![Deepgram logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2020/12/WRILS-201212160008-406229.jpg) Deepgram Oct 29 New ### [Billing & Analytics Software Engineer](https://jobicy.com/jobs/157871-billing-analytics-software-engineer.md) Company Overview Deepgram is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing real-time APIs for speech-to-text (STT), text-to-speech (TTS), and building production-grade voice agents at scale. More… * ![hims & hers logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2023/11/e7b9b920-221.jpeg) hims & hers Oct 29 New ### [Sr. Software Engineer, (Fullstack/Frontend)](https://jobicy.com/jobs/157866-sr-software-engineer-fullstack-frontend.md) Hims & Hers is the leading health and wellness platform, on a mission to help the world feel great through the power of better health. We are redefining healthcare by… * ![Deepgram logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2020/12/WRILS-201212160008-406229.jpg) Deepgram Oct 29 New ### [Backend Software Engineer – Engine Team (Voice Agent)](https://jobicy.com/jobs/157859-backend-software-engineer-engine-team-voice-agent.md) Company Overview Deepgram is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing real-time APIs for speech-to-text (STT), text-to-speech (TTS), and building production-grade voice agents at scale. More… * ![Deepgram logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2020/12/WRILS-201212160008-406229.jpg) Deepgram Oct 29 New ### [Research Engineer, Machine Learning Systems](https://jobicy.com/jobs/157855-research-engineer-machine-learning-systems.md) Company Overview Deepgram is the leading platform underpinning the emerging trillion-dollar Voice AI economy, providing real-time APIs for speech-to-text (STT), text-to-speech (TTS), and building production-grade voice agents at scale. More… * ![hims & hers logo](https://jobicy.com/data/server-nyc0409/galaxy/mercury/2023/11/e7b9b920-221.jpeg) hims & hers Oct 29 New ### [Sr. Staff Engineer, Care (Fullstack/Backend)](https://jobicy.com/jobs/157848-sr-staff-engineer-care-fullstack-backend.md) Hims & Hers is the leading health and wellness platform, on a mission to help the world feel great through the power of better health. We are redefining healthcare by… [Browse all jobs](https://jobicy.com/jobs.md)